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ABSTRACT 

A longitudinal study was conducted in three Maryland 
counties to determine the predictive validity of the Maryland 
Systematic Teacher Observation Instruirent (HSTOI) and other initial 
assessment instruments as screening devices to identify kindergarten 
children with potential reading and/or learning difficulties. The 
BSTOI was also examined to determine its longitudinal validity as a 
predictor of children with reading an'!/or learning problens. 
Information collected in each county included data en initial tests, 
follow-up tests, concurrent validity, fredictive validity, item 
analysis, and a discussion of procedures and results. It was 
determined that the predictive validity of KSTOI compared favorably 
with other initial assessment instruments. Fredictive validity data 
on HSTOI wa*" developed and its relationship to student progress and 
retention was established. Recommendations for further study are 
suggested. Summary tables for individual counties and combined data, 
descriptions of assessment instruments, and technical notes arc 
appended. (M H) 
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Introduction 

Drior t the int^oc..- of House Bill 234 in the Maryland 
State Legis diare, the *»*ry'^d State Depar~ent of Education 
organ red a wooing rc ^^^-^ence to investigate practical w^rvs for 
educaiXD-s to i^ETrci^'y* helo children with potential leserr^t:' 
readim; prcjoleiiLS, 

of the T2cric~^ s^f the conference, ami of subseqinent 
Warylcr State J^arrinff^ ^ Education efforts, was the develapmejnt 
of a tr : plan t : e^c 'eztm local educational agencies -o in^l^fnet^^t 
the prctniions of Se t err- 5SC, Article 77 of rhe Annotated Coe c 
Maryland The ir>'>sic : in, developed cooperatively with the Iqc*^^ 
agency;^ -ly lale laf'^on Coordinators, has three components ^dllinc 
for: r) early/ Screentnn^ of all students, (2) administering 
continunus as<:«>?ssiient -students, and (3) developing instructional 
stratejfes basecr c n t!' screening and/or assessment findings. 

It i^as apparent t ill developers of the Early Identification 
Program that '?tssessme'^'t instruments of good quality were essential 
to tfe -,ucce^^ if the urogram. An ideal assessment instruaent 
shouliti - val c:, reliable, easy to administer, and easily ^cored. 
Sinei- exisrrrng evaluative measure possessed all of the^^ charac- 
ter!: to ^n ideal level, certain compromises would hav^ to be 
made i^' ^.e selection of assessment measures. However, program 
sucteio oulc be enhanced through careful planning of the ai^ess- 
ment comment. Therefore an overall test strategy was deve^loped 
that requr^d periodic evaluation of the program at different :Dhases 
in thr impleimentation of the Early Identification Program. B^h 



evaluation phase logically snd systematical], reviewed a portion -f 

the assessment phase. 

In th& first phase, art evaluation study was performed that 
reported ufum the develiTpnEsnt and validatton of a systematic trar^ 
observatiofl -nstrument -975). It was -eported that the 
systematic - "sicher observat-.on instrument vitas valid and could be 
user? zo d-srrnguish betwesn r yrmal children and special children -rr 
the .-Tirin^-gsTten and in "tne "St grade. An important outcome of 
thi ^ "tiuarion was the establishment of cutoff scores for the 
sysT .lu c teacher observa: or instrument. A second portion of 
th— St' / reported the development and validation of a parent 
ir 3rv^r<- checklist. An a^-'l sis of the data indicated that the 
pe-rent i'>terview checklis - its present form was not useful for 
d-=tingu shing between no ai and special children. The final 
P'vv^^ion Tf the study ider ied and evaluated a number of stan- 
ds J i zed tests that were -U-eful in identifying normal children 

children with potent: learning disabilities in the kinder- 
garten and grade one. 

In the second phase, several aspects of the assessment portion 
of the Early Identification Program were examined (2, 1976). First, 
the reliability of the N'.aryland Systematic Teacher Observation 
Instrument was determined, using the test-retest procedure. It 
was concluded that this instrument is reliable across time when 
used by the same teacher. A second area examined was the validity 
of a revised Parent Interview Checklist . An analysis of the data 
indicated that the Parent Interview Checklist was not valid for 
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identifying children with potential learning difficulties Tins 
evaluation study reported that teachers using the Marylard Sys- 
tematic Observation Instrument were identifying approximsiiely 
40% of the students as high risk students. This figure ciiosBly 
corresponds to tSie hypothesized figure of 40%, Finally, ziuring 
this phase, a three year longitudinal study was initiatec tc 
determine the predictive validity of the Maryland Systemgri:: 
Teacher Observation Instrument and selected standardized £ts 
that were administered to children in the kindergarten. 

The two studies above were primarily concerned with ue 
development of an instrument and assessing the instrumerx'3 
ability to identify special students. The present stud^> was 
concerned primarily with determining the predictive val idity 
of the MSTOI and the other assessment instruments for 
population where there had been no intervention based ^n MSTOI 
performance. 

Methodology 

Purpose of the Study 

The purpose of this longitudinal study was to determine the 
predictive validity of the Maryland Systematic Teacher Observation 
Instrument and other initial assessment instruments. The study 
specifically focused on a detailed analysis of the Maryland Sys- 
tematic Teacher Observation Instrument (MSTOI) as a screening 
measure for identifying kindergarten children with potential 
reading/learning difficulties and as a predictor of continued 
identification of these children as having reading/learning problems 
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Procedure and SanEL.-^ Description 

The studi* wes conducted in Queen Anne's County, Wicomico 
County, and Ce-ii -zjunty. These counties were selected on the 
basis of their a^rtion in the Maryland State Department of 
Education plan for implementing the Early Identification and 
Intervention ^'rogram. Since these counties were not scheduled 
to implement ^ program before the completion of the study, it 
was possible Tto assess the predictive validity of the MSTOI in 
the absence cpt an intervention program based on diagnostic data. 

The MSTGI was administered to all kindergarten children in 
these counti;:es during the 1974-75 school year. The tests were 
administered atid scored by the kindergarten teachers. In Queen 
Anne's County, the MSTOI was administered to 309 youngsters along 
with the Cognitive Skills Assessment Battery (CSAB) and the Class- 
room Behavior Inventory (CBI). In Wicomico County, the MSTOI 
was administered to 550 youngsters along with the CSAB and the 
CBI. In Cecil County, 435 MSTOI evaluations were made along 
with the administration of the Meeting Stre et School Screening 
Test (MSSST) and S elf Control Behavior Inventory (SCBI) to 100 
individuals. Each of these tests is described in Appendix B. 

For the initial assessment instruments, cutoff scores were 
used to differentiate between those students judged as successful 
and those identified as having potential reading/learning problems. 
These cutoff scores were established in phase one of the evaluation 
program (1, 197 5). For the MSTOI, a cutoff score of 139 was 
established. This score represented an average of the mean scores 
made by the normal and special groups in the norming sample. Any 

9 
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student scoring less than 139 was classified as having potential 
reading/learning problems and subject to further testing. In 
establishing cutoff scores for the standardized tests used in the 
initial assessment, the procedure used was slightly different from 
the procedure used with the MSTOI . Cutoff scores for the CSAB, 
MSSST, CBI, and SCBI were established by first determining the 
mean on each test for the high and low sample group. For the 
low group, one standard deviation was added to the mean; for the 
high group, one standard deviation was subtracted from the mean. 
The higher of these two scores was established as the cutoff score. 
Using this procedure, the cutoff scores established were: 128 
for the Cognitive Skills Assessment Battery , 58 for the Meeting 
Street School Screening Test , 32 for the Classroom Behavior Inventory , 
and 15 for the Self Control Behavior Inventory . A student scoring 
below these cutoff scores was identified as having a potential 
reading/learning problem. 

The previously established cutoff scores were used for classi- 
fying students and for comparing performances on the various 
assessment measures. The criterion scores were conservative, since 
it was thought to be beneficial to over Identify potential problems 
rather than miss some that might need attention. This rationale 
was justified on the basis that youngsters so identified were not 
to be placed until given further testing. 

In the Spring of 1977, when the average subject was completing 
second grade, the follow-up criterion tests were administered to 
many of these youngsters. The two criterion instruments used were 
the Pupil Rating Scale (PRS) and the Peabody Individual Achievement 
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Test (PIAT). The PRS was used by the classroom teachers to evaluate 
the students. Since the PIAT is an individually administered test, 
one examiner in the county administered and scored the test. 

Cutoff scores for the PIAT and the PRS were established by 
the evaluation consultants. A cutoff score of 102 was established 
for the PIAT and a score of 62 for the PRS. These scores represent 
a score one standard deviation below the mean of the national norms 
as supplied by the test publisher. The percentage of youngsters 
identified in this manner would be the lowest 15% of the national 
population. The guidelines from the Maryland State Department of 
Education indicated that approximately 15% of the students would 
require individualized services (3, 1975). The criterion measures 
would help to identify a group of comparable size. 
Analysis 

The data analysis was divided into three stages: (1) deter- 
mination of concurrent validity, (2) determination of predictive 
validity, (3) determination of MSTOl item analysis. Assessment 
of concurrent validity and predictive validity was done by employing 
two procedures, each having certain advantages. The use of cutoff 
scores has practical advantages because it can be used to classify 
students. With cutoff scores, contingency tables can be developed 
that enable the calculation of to determine the existence of a 
significant relationship between the groups identified by the two 
different assessment instruments. After computing the chi -square 
value, the magnitude of the relationship could be determined by 
computing phi. A shortcoming of these statistical techniques 
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is the sensitivity to the cutoff score selected. That is, different 
values for cutoff scores produce varying values for chi-square 
and phi. Therefore, it was appropriate to examine the relationship 
between the two assessment instruments utilizing a procedure free 
from the shortcoming of the cutoff score. The Pearson Product 
Moment Correlation Coefficient was selected as an appropriate measure 
to assess the strength of the relationship between the total test 
scores as well as the relationship among the various subscores. 

In the preliminary stage, the concurrent validity of the MSTOI 
was determined. Concurrent validity was assessed by determining the 
contingency coefficients and the Pearson Product Moment Correlation 
Coefficients between the MSTOI and the four other initial assessment 
instruments. The data in the study made it possible to determine 
ivr-Mjryptnt validity with larger samples than had been used in the 
■ validation studies of the MSTOI. 

major stage of the analysis was the determination of the 
predictive validity of the initial assessment instruments. Chi- 
sqt'^are values were determined for each of the five initial assessment 
instruments using each of the two criterion measures. The reasons 
for employing two statistical techniques to determine validity are 
as stated previously. 

Further analysis was performed on each MSTOI item to determine 
its relationship with: (1) the MSTOI total scores, (2) the MSTOI 
subscores, and (3) the total scores of the criterion tests. 

By design, the set of tests used and the sampling procedures 
employed in each county were different. Results from Queen Anne's 
County will be presented first followed by Wicomico County and then 
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Cecil County. Results that are applicable to more than one county then 
will be reported and discussed. 

Results - Queen Anne's County 

Initial Tests 

A total of 309 MSTOI tests was administered to kindergarten youngsters 
during the 1974-75 school year. From this group, the total of completed 
usable test forms was 305. The mean of the total score for the group 
was 145.13 with a standard deviation of 20=27. An analysis of the 
individual items for this initial sample is shown in Appendix A in Table A. 
(All lettered tables appear in Appendix A.) 

In addition to the MSTOI, the CSAB was administered to each youngster. 
As shown in Table B, the mean of the total score was 108.40 and the 
standard deviation was 17.04. 

At approximately the same time, these youngsters were rated using 
the Classroom Behavior Inventory . Valid observations were obtained on 
308 youngsters. As shown in Appendix A, Table C, the mean for the total 
score was 29.94 with a standard deviation of 7.54. 
Follow-up Tests 

In the Spring of 1977, two criterion measures, the Peabodv Individual 
Achievement Test (PIAT) and the Pupil Rating Scale {PRS),were administered 
to a sample of students. Since the PIAT required individual testing, a 
limited number of children were tested. Children were selected on a 
random basis and a total of 80 children were tested. The results of this 
are shown on Table D, Appendix A. The total test mean was 150.79 with a 
standard deviation of 33.24. 

An attempt was made to locate all members of the initial sample 
and to administer the PRS to them. The classroom teachers rated 230 of 
the original sample with the PRS. The results are presented in Appendix A. 

13 
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Table E. The mean of total scores was 74.16 and the standard deviation 
was 15.99. Values for subscales are also listed in the table. 

In order to determine whether the follow-up sample was representa- 
tive of the initial sample, the initial mean for MSTOI scores was 
compared to the follow-up MSTOI scores. The mean for the MSTOI follow- 
up sample was 145.80 with a standard deviation of 20.60. A z-test was 
performed and it was concluded that there was no significant difference 
between the initial and follow-up samples on the MSTOI total scores. 
Means and standard deviations of the MSTOI, CSAB, CBI follow-up samples 
are shown in Appendix A, Tables F-H. 
Concurrent Validity 

Utilizing the data from the follow-up sample, the concurrent 
validity of the MSTOI with the CSAB and the CBI was determined in two 
ways. Test scores were classified as either above or below the cutoff 
point for the test. In order to test the relationship between perfor- 
mance on two different tests, a contingency table was then developed. 
This table showed how many were above the cutoff point on both tests, 
below the cutoff point, or above on one test and below on the other. 

This inTOrtlldtlu:! Was uiaH aPaiyicQ D> .-nv. — - t— 

significant relationship or degree of association between the two tests. 
Phi coefficients and Pearson Product Moment Correlation Coefficients 
were calculated. As shown in Table 1, no student with a MSTOI score of 
less than 139, scored 128 or greater on the CSAB. 
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■raBLE 1 



Distribution Of Queen Arrne's County Follow-up Sample 
Classified By MSTOI Parformance And CSAB Total Scores 



MSTOI Total Score 



CSAB Total Score 



139 or Greater 



Less than 139 



Column Total 



128 


or Greater 


Less than 128 


Row Total 


N 


24 


138 


162 


Row % 


15 


85 


71% 


Col. % 


100 


68 


N 


0 


65 


65 


Row t 


0 


100 




Col . % 


0 


32 


29% 


N ■ 


24 


203 


227 


% 


11 


89 


100 



Chi-square = 9.26 
Phi = .22 



P < .01 



In Table 2, as in Table 1, the calculated value indicates that 
there was a significant degree of association between the MSTOI total 
scores and the other initial assessment instrument. 

TABLE 2 



Distribution Of Queen Anne's County Follow-up Sample 
Classified By MSTOI Performance And CBI Total Scores 


MSTOI Total Score 




CBI Total 


Score 






32 


or Greater 


Less than 32 Row Total 


139 or Greater 


N 

Row % 
Col. % 


109 
67 
94 


55 
33 
48 


164 

71% 


Less than 139 


N 

Row % 
Col . % 


7 

11 
6 


59 
89 
52 


66 

29% 


Column Total 


N 

% 


116 

50 


114 

50 


230 
100 



P ^ .01 



Phi = .51 



is 
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The Pearson Product Motnes^ Correlation Coefficients were determined 
between the MSTOI and the CSAB and the CBI. Total score and subscale 
coefficients were significant at the .01 level. These data are shown in 
Table 3. 

TABLE 3 

Pearson Product Moment Correlation Coefficients Between MSTOI And 
The CSAB And The CBI For The Queen Anne's County Follow-up Sample 



MSTOI Subscale 


CSAB Total 
N=227 


CBI Total 
N=230 


PsychuhiOtOf 


cn 

• 


.65 


Language 


.73 


.70 


Sensory/Perception 


.71 


.66 


Cognition 


.64 


.65 


Affect/Motivation 


.56 


.67 


MSTOI Total 


.72 


.73 



Note: All correlations significant at the ,01 level. 



The two analyses of the relationship between the MSTOI scores and 
the two other initial assessment instruments indicate that the MSTOI 
possesses concurrent validity. Using the MSTOI, teachers can assess 
the same behaviors assessed with standardized tests, the CSAB and the 
CBI. 

Predictive Validity 

The same statistical techniques were used to determine predictive 
validity as were used to determine concurrent validity. The criterion 
tests employed were the PIAT and the PRS. Tables 4-7 indicate the 
degree of relationship between the MSTOI and the criterion measures. 

16 
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As shown in Table 4, a nonsignificant relationship between the 
MSTOI score classification and the PIAT score classification was found. 
It is interesting to note ttet 95% of the PIAT scores were above the 
cutoff level . 



TABLE 4 

Distribution Of Queen Anne's County Follow-up Sample Classified 
By MSTOI Performance And PIAT Total Scores (N=80) 



MSTOI Total Score 



139 or Greater 



Less than 139 



Column Total 



Chi-square = 2.86 
Phi = .25 



N 

Row % 
Col. % 

N 

Row % 
Col. I 

N 



PIAT Total Score 



102 or Greater Less than 102 Row Total 



58 
98 

76 

18 
86 
24 

76 
95 



1 
2 
25 

3 

14 

75 

4 

5 



P = Not Significant 



59 

7456 

21 

2656 

80 
100 



As shown in Table 5, a significant correlation exists between the 
MSTOI and PIAT scores. The reader should note that the language sub- 
scales have at least as strong a relationship with the PIAT scores as 
the MSTOI total scores do. 
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TABLE 5 

Pearson Product Moment Correlation Coefficients Between MSTOI And 
PIAT Scores For Queen Anne's County Follow-up Sample (N=80) 

MSTOI Subscales ' PIAT Subscales 





Mathematics 


Reading 
Recognition 


Reading 
Comprehension 


Spelling 


General 
Information 


PIAT Total 


Psychomotor 


.35 


.28 


.35 


.26 


.35 


.40 


Language 


.39 


.35 


.44 


.35 


.42 


.50 


Sensory/Perception 


.36 


.35 


.45 


.35 


.35 


.47 


Cognition 


.28 


.29 


.38 


.29 


.30 


.39 


Affect/Motivation 


.28 


.18* 


.28 


.24* 


.26 


.32 


MSTOI Total 


.36 


.28 


.38 


.31 


.35 


.43 


Note: All correlations significant at the .Ui level except inose 
marked *. 

Table 5 shows the results of the analysis which indicates that 
the relationship between the MSTOI and PRS score categories is signifi 
cant at the .01 level. This is in contrast with results of a nonsig- 



nificant relationship noted between the MSTOI and PIAT score categories. 
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TABLE 6 



Distribution Of Queen Anne's County Follow-up Sample 
Classified By MSTOI Performance And PRS Total Scores 



MSTOI Total Score PRS Total Score 



139 or Greater N 

Row % 
Col. % 

Less than 139 N 

Row % 

t^~1 0/ 



Column Totals N 

% 



52 or Greater 


Less than 62 


Row Total 


146 


18 


164 


89 


n 




80 


38 


71% 


36 


30 


66 


55 


45 




20 


62 


29% 


182 


48 


Z30 


79 


21 


100 



Chi -square = 31.82 P ^ -01 

Phi = .38 

The correlation between the MSTOI total score and the PRS total 
score was .53. Correlations among the PRS and MSTOI subscores are shown 
in Table 7. In this analysis also, the language subscore has as strong 
a relationship as the MSTOI total scores with PRS subscores. 
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TABLE 7 

Pearson Product Moment Correlation Coefficients Between The 
M-JOI And PRS For Queen Anne's County Follow-up Sample (N=239) 



MSTOI Siibscores PRS Subscores 



to 

coo 

to o ^ 

C .r- +J »- • 

%J SI CD ro c o :J 

O Q) C fO -P -r- C -r- J- ^ 

0)3 c s-xj o> 2 5-' — 

•r-CL -2ZCn 0) OS- tofO -p 

-n£^ OC -r- +JO i-JC S- S^. OO 

30 Q-rO OO CUOJ OJ O CC 

<c o CO -J o s: o D- CO > z: a. 



Psychomotor .43 .46 .48 .43 .52 .46 .51 .52 

Language .56 .56 .53 .44 .55 .58 .53 .59 
Sensory/ 

Perception .50 .49 .51 .40 .51 .51 .50 .53 

Cognition .45 .41 .42 .32 .45 .44 .42 .45 

Moti^*rior .36 .36 .43 .43 .56 .37 .51 .48 

MSTQi- Total .48 .48 .47 .42 .55 .49 .51 .53 



Wuf: All correlations significant at the .01 level. 

As shown in Table 8, a nonsignificant relationship ves found between 
the CSAB classification and the PIAT score classification. Paradoxically, 
96% of the PIAT scores were above the cutoff point, whereas for the same 
group 90% of the CSAB scores were below the cutoff point. 
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TABLE 8 



Distribution Of Queen Anne's County Follow-up Sample 
Classified By CSAB Performance And PIAT Total Scores 



CSAB Total Scores 



PIAT Total Scores 



138 or Greater 



Less than 138 



Column Total 



N 

Row % 
Col. % 

N 

Row % 

Col . % 

N 

% 



102 or Greater 


Less than 102 


Row Total 


8 


0 


8 


100 


0 




; 11 


0 


10% 


68 


3 


71 


96 


4 




; 89 


100 


90% 


76 


3 


79 


96 


4 


100 



Chi -square 
Phi = .07 



,14 



P = Not Significant 



The CSAB assesses five different competency areas. The correlation 
coefficients indicating a relationship among the CSAB and PIAT competency 
areas are shown in Table 9. A significant relationship was observed 
between the CSAB and PIAT total scores. 
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TABLE 9 

Pearson Product Moment Correlation Coefficients Between 
CSAB And PIAT For Queen Anne's County Follow-up Sample (N-79) 



CSAB 

Competency 
Areas 



PIAT Subscores 





Mathematics 


Reading 
Recognition 


Reading 
Comprehension 


Spel 1 ing 


General 
Information 


PIAT Total 


Orientation 


.29 


.25* 


.46 


.24* 


.34 


.40 


Coordination 


.23* 


.31 


.39 


.24* 


.15** 


.33 


Discrimination 


.41 


.46 


.47 


.36 


.38 


.53 


Memory 


.44 


.37 


.45 


.29 


.47 


.52 


Comprehension/ 






.50 


.28 


.54 


.55 


Concept 


.43 


.38 


CSAB Total 


.44 


.41 


.55 


.31 


.48 


.56 


Note: All unstarred are 


significant at the 


.01 level. 





*Significant at .05 level. 
**Not significant. 



The predictive validity of the CSAB was determined, using the PRS 
as the criterion measure. Table 10 documents the results of classifying 
PRS and CSAB scores by the cutoff points. The calculated indicates a 
relationship significant at the .05 level. 
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TABLE 10 



Distribution Of Queen Anne's County Follow-up Sample 
Classified By CSAB Performance And PRS Total Scores 

CSAB Total Score PRS Total Score 



128 or Greater N 

Row t 
Col. % 

Less than 128 N 

Row % 



Col. % 



Column Total N 

% 



62 or Greater 


Less than 62 


Row Total 


24 


0 


24 


100 


0 


10 


13 


0 


158 


47 


205 


77 


23 


90 


87 


100 


182 


47 


229 


80 


20 


100 



Chi-square = 5.59 P ^ -05 

Phi = .17 

Correlation coefficients between the CSAB and PRS were calculated, 
The correlation of total CSAB with total PRS was .58. Correlations 
of the competency areas and the PRS scores are shown in Table 11. 

TABLE 11 

Pearson Product Moment Correlation Coefficients Between CSAB 
And PRS For Queen Anne's County Follow-up Sample (N-229) 



CSAB 

Competency 
Areas 


PRS 
Verbal 


Subscore 

Nonverbal 


PRS Total 


Orientation 


.54 


.44 


.52 


Coordination 


.35 


.35 


.35 


Discrimination 


.49 


.39 


.46 


Memory 


.59 


.49 


.55 


Comprehension/ 
Concept 


.60 


.46 


.54 


CSAB Total 


.62 


.50 


.58 



Note: All correlations significant at .01 level 
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The predictive validity of the CBI was determined using the PIAT 
total scores as the criterion measure. These data are shown in Tables 
12-15. 



TABLE 12 

Distribution Of Queen Anne's County Follow-up Sample 
Classified By CBI Performance And PIAT Total Scores 



CBI Total Scores 




PIAT Total 
102 or Greater 


Scores 

Less than 102 


Row Total 


32 or Greater 


N 


46 


0 


46 




Row % 


100 


0 






Col . % 


61 


0 


58% 


Less than 32 


N 


30 


4 


34 




Row % 


88 


12 






Col. % 


39 


100 


42% 


Column Total 


N 


76 


4 


80 




% 


95 


5 


100 



Chi-square = 3.49 
Phi = .27 



P = Not significant 



The predictive validity of CBI was further assessed by determining 
the correlation between CBI total scores and the PIAT scores. The results 
are shown in Table 13- 

TABLE 13 

Pearson Product Moment Correlation Coefficients Between The CBI Total 
Score And PIAT Scores For Queen Anne's County Follow-up Sample (N=80) 

■ PIAT Subscores ' 



EKLC 



01 

JC 



CBI Total 



.32 



c 
o 



c c 

•r- cn 

•a o 

fO u 



.27 



o 

•r— 

to 
c 

(D 

cn-c 

C CD 
•r- 

-a dl 

ro E 
(D O 



.36 



cn 

r: 



(D 

Dl 
CO 



.27 



o 



(D O 

c ^- 

CD I— I 



.32 



Note: All correlations significant at .01 level. 

2i 



o 
I— 

I— 

I— I 

a. 



.39 
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Table 14 and 15 summarize the results of the predictive validity 
calculations for the CBI with PRS as a criterion measure. 

The X2 value obtained indicated a significant degree of relation- 
ship between the CBI and PRS score distribution. Table 14 shows the 
score distribution. 

TABLE 14 

Distribution Of Queen Anne's County Follow-up Sample 
Classified By CBI Performance And PRS Total Scores 



CBI Total Scores PRS Total Scores 



32 or Greater N 

Row % 
Col. % 

Less than 32 N 

Row 7o 
Col. % 

Column Total N 

% 

Chi-square = 35.41 
Phi = .41 



62 or Greater 


Less than 62 


Row Total 


111 


5 


11 6 


96 


4 


50% 


61 


10 


72 


43 


115 


63 


37 




39 


90 


50% 


183 


48 


231 


79 


21 


100 


P ^ .01 



The final calculation dons in order to assess the predictive 
validity of the CBI was to calculate its correlation with the PRS. As 
shown in Table 15, a significant relationship between these measures 
was observed. 



TABLE 15 



Pearson Product Moment Correlation Coefficients Between CBI Total 
Score And PRS Score For Queen Anne's County Follow-up Sample (N=<J3U 



PRS Subscales 



Verbal Nonverbal Total 

CBI Total .49 .53 .69 

Notel All correlations significant at .01 
level . 
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Item Analysis 

In order to assess the predictive validity of the MSTOI, an item 
analysis was performed. Each MSTOI item and MSTOI subscale was compared 
to the MSTOI total score. Each MSTOI item and subscore was compared 
to the total score of each of the criterion measures. These data are 
shown in Tables 16 and 17. 
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TABLE 16 



MSTOI Item Analysis Correlations Of Each Item 
And MSTOI Total Score For Queen Anne's County 



With MSTOI Subscales 
Follow-up Sample (N=230) 



MSTOI Item 



MSTOI Subscales 





Psychomotor 


Language 


Sensory/ 
Perception 


C 
0 
•r- 
+J 
•r- 

a> 

Q 

0 


Affect/ 
Motivation 


MSTOI Total 


1 


7/1 
• # "1 


.57 


.66 


.49 


. 30 


. DO 


2 


.71 


.73 


.74 


.67 


. /U 


77 


3 


.57 


.78 


.73 


.80 


.bl 


7/1 


4 




.66 


.74 


.84 


.55 


.75 


5 


.50 


!49 


.40 


.34 


.57 


.54 


6 


.58 


.73 


.55 


.51 


.46 


.62 


7 


.48 


.50 


.36 


.35 


.61 


.52 


8 


.74 


.52 


.77 


.73 


.54 


.69 


9 


.80 


.69 


.73 


.67 


.80 


.80 


1 u 


.72 


.88 


.78 


80 


.66 


.85 


11 


.75 


.80 


.74 


.71 


10 


. ou 


12 


.68 


.79 


.69 


.73 


.0 1 


7R 


13 


.37 


.24 


.28 


.37 


/I Q 

.4b 


"iQ 

. jy 


14 




/in 




.65 


.46 


.56 


15 


.36 


.20 


.21 


.16 


.66 


.37 


16 


.50 


.63 


.74 


.83 


.44 


.69 


17 


.63 


.65 


.69 


.80 


.55 


.73 


18 


.46 


.40 


.33 


.36 


.55 


.48 


19 


.62 


.57 


.68 


.81 


.53 


.70 


20 


.53 


.60 


.53 


. O'f 


.60 


.62 


21 


.67 


.59 


.73 


.60 




79 


22 


.44 


.23 


.28 


.21 


. Do 


. He 


23 


.63 


.63 


.41 


.36 


A "7 


. bJ 


24 


.74 


.81 


-7-7 

.11 


.71 


. / D 


• 04 


25 


.46 


.23 


.30 


.25 


.71 


.45 


26 


.58 


.57 


.73 


.78 


.52 


"7 f\ 

.70 


27 


.56 


.43 


.51 


.49 


.74 


.62 


28 


.60 


.85 


.73 


.74 


.55 


.77 


29 


.49 


.36 


.30 


.33 


.46 


.44 


30 


.63 


.59 


.81 


.77 


.48 


.70 


31 


.59 


.60 


.71 




.48 


.70 


32 


.59 


.68 


.74 


.83 


.53 


.74 


33 


.65 


.68 


.57 


.59 


.61 


.69 


34 


.61 


.64 


.74 


.53 


.51 


.66 


35 


.51 


.76 


.68 


.53 


.49 


.67 


36 


.49 


.76 


.63 


.50 


.47 


.64 


Psychomotor 




.81 


.85 


.76 


.82 


.91 


Language 






.84 


.82 


.72 


.92 


Sensory/Percepti on 






.89 


.71 


.93 


Cognition 










.67 


.90 


Affect/Motivation 










.88 


Note: All 


correlations significant at the 


.01 



level 
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TABLE 17 

MSTOI Item Analysis 
Correlations Of Each MSTOI Item And Subscale With The Total PIAT (N=80) And 
PRS (N=230) Criterion Measures For Queen Anne's County Follow-up Sample 



MSTOI Item MSTOI Subscales 



o c c >— 

4-> O C O (O r— 

O <I> "-^"i- O •!- +J (O 

e cr» >,+j •!- o +-> 

O «0 S-Ol+-> •♦JfO I— p 

£3 0(U-r- 0> I— 

>, C CS- C7> <^ ■*-> et to 

«/» <0 0)0) O *4-0 I— I 

O; =J CO Q- <-> <c s: ^ ^ 

1 .74 .57 .66 ,49 .58 ,31 ,42 

2 .71 .73 .74 .67 .70 .31 .51 

3 .57 .78 .73 .80 .51 .43 .48 

4 .65 .66 .74 .84 .55 *.23 .30 

5 .50 .49 .40 .34 .57 **-.02 .36 

6 .58 .73 .55 .51 .46 .35 .42 

7 .48 .50 .36 .35 .61 **.15 .23 

8 .74 .52 .77 .73 .54 .37 .29 

9 .80 .69 .73 .67 .80 .29 .51 

10 .72 .88 .78 .80 .66 .45 .48 

11 .75 .80 .74 .71 .72 .40 .57 

12 .68 .79 .69 .73 .61 .31 .38 

13 .37 .24 .28 .37 .45 **.15 .05 

14 .53 .40 .53 .65 .46 **.10 .14 

15 .36 .20 .21 .16 .66 *.21 .17 

16 .50 .63 .74 .83 .44 .43 .37 

17 .63 .65 .69 .80 .55 **.18 .27 

18 .46 .40 .33 .36 .55 **.02 .27 

19 .62 .57 .68 .81 .53 *.20 .30 

20 .53 .60 .53 .54 .60 .30 .38 

21 .67 .59 .73 .60 .64 .29 .32 

22 .44 .23 .28 .21 .68 *.20 .19 

23 .63 .63 .41 .36 .47 .37 .32 

24 .74 .81 .77 .71 .76 .37 .58 

25 .46 .23 .30 .25 .71 **.ll .15 

26 .58 .57 .73 .78 .52 *.23 .26 

27 .56 .43 .51 .49 .74 **.ll .29 

28 .60 .85 .73 .74 .55 .37 .55 

29 .49 .36 .30 .33 .46 **-.04 .23 

30 .63 .59 .81 .77 .48 .34' .30 

31 .59 .60 .71 .85 .48 .28 .27 

32 .59 .68 .74 .83 .53 *.24 .40 

33 .65 .68 .57 .59 .61 *.24 .29 

34 .61 .64 .74 .53 .51 .43 .50 

35 .51 .76 .68 .53 .49 .33 .54 

36 .49 .76 .63 .50 .47 .35 .51 
Psychomotor .81 .85 .76 .82 .40 .45 
Language .84 .82 .72 .50 .53 
Sensory/Perception .89 .71 .47 .48 
Cognition •67 .39 .39 
Affect/Motivation -32 .41 

MSTOI Total -43 .53 

All unmarked correlations significant at the .01 level, 

O *Significant at the .05 level. 
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Discussion 

A summary of the validity indices for the initial assessment 
instruments is shown in Table 18. 

TABLE 18 

Validity Indices For The MSTOI, CSAB And CBI 
For Queen Anne's County Follow-up Sample 







r 


Concurrent Validity 




.72 


MSTOI with CSAB 


9.26 


MSTOI with CBI 


56.52 


.73 


Predictive Validity 




.53 


MSTOI with PRS 


31 .82 


CSAB with PRS 


5.59* 


.58 


CBI with PRS 


36.41 


.69 


MSTOI with PI AT 


2.86** 


. .43 


CSAB with PIAT 


.14** 


.56 


CBI with PIAT 


3.49** 


.39 



**Not significant at the .05 level. 
*Significant at the .05 level. 
All other values significant at the .01 
level . 



While interpreting the data in Table 17, the reader should be 
cognizant of certain statistical assumptions and constraints. In this 
study the x2 was used primarily for testing for the relationships between 
dichotomized score distributions. The distributions tested were indicated 
by the student performance on various assessment instruments. Obviously 
the cutoff score used will influence the magnitude of the obtained X2. 
An advantage associated with this procedure is that students are classified 
as a teacher would be required to classify them when interpreting MSTOI 
test results. In addition, the contingency table, an initial step in 
calculating X^, provides a graphic presentation of score distributions. 



29 



25 

On the other hand, the Pearson Product Moment Correlation Coefficient 
procedure gives a more precise estimate of the degree of relationship 
between the various assessment instruments. Since the Pearson Product 
Moment Correlation Coefficient utilizes the entire continuum of scores, 
it is not cutoff score dependent. Further, Pearson Product Moment 
Correlation Coefficients can be used to determine relationships among 
various subscores. Pearson Product Moment Correlation Coefficients 
also can be analyzed to determine the existence of a statistically 
significant difference between correlation coefficients. 

The results presented in this table are based on data from one 
sample and caution should be exercised when generalizing results. 

In assessing the concurrent validity, it was noted that the MSTOI 
related as strongly to the CSAB as to the CBI. As might be expected, 
the concurrent validity coefficients are higher than the predictive 
validity coefficients. This normally is the case since there are 
fewer intervening variables over a shorter period of time. 

When examining the predictive validity coefficients, it Is 
interesting to note that all of the Pearson Product Moment Correlation 
Coefficients are significant at the .01 level. Whereas none of the X2 
relationships between the initial assessment instruments and the PIAT 
are significant. A possible explanation for this apparent inconsistency 
may be the selection of inappropriate cutoff scores. For example, the 
MSTOI score was established to select the bottom 40% of those tested 
for further attention. In the Queen Anne's County follow-up sample, 
21% scored below the cutoff score. For the PIAT, only 5% scored below 
the cutoff while on the other criterion test, the PRS, 21% scored below 
the cutoff. It must be noted that the various cutoff scores were 
selected as standards for the State of Maryland and these anomalous 
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results may have been due to factors indigenous to this county. 

A t-test for testing for differences between dependent correlations 
revealed that only the predictive validity coefficient for the CBI with 
the PRS was significantly higher than the MSTOI predictive validity 
coefficient (t=2.95). In other words, the MSTOI predictive validity 
coefficient was as good as the validity coefficients of the CSAB and 
the CBI in three out of four comparisons. Although all of the validity 
correlation coefficients are significant at the .01 level or better, 
it must be noted that the chi-square (X^) values were not always 
significant. This seems especially evident in the CSAB and PIAT analyses. 
A possible explanation for this apparent inconsistency may be the selec- 
tion of inappropriate cutoff scores used in the contingency table. 

A rather intensive item and subscale analysis of the MSTOI was 
performed. The results may be sunmarized as follows: 1) all of the 
subscales had predictive validity coefficients significant at the .01 
level for both criteria (PIAT and PRS total scores), 2) all individual 
items were related significantly to the total MSTOI score and to at 
least one criterion total score (in general the items related 
significantly with both criteria), 3) the relationship between the 
MSTOI items and the PRS was apparently higher than for the PIAT. 

The intercorrelations among the five subscales are striking 
in that the majority are in the .80 to .89 range. This degree of 
interrelatedness is reflected in the generally consistent size of 
correlation coefficients for a particular MSTOI item. This is true 
with each of the five subscales. There are several possible explana- 
tions for these findings. One reason may be that the various skills 
measured by the MSTOI are very highly related and therefore one would 
expect a high degree of interrelatedness among the various items and 
Q subscales. Another possibility is that the "halo effect" reduces the 
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variability among item ratings. For example, the teacher that has an 
overall impression of the child may tend to rate the student similarly 
on all items. The extent to which these explanations are valid cannot 
be determined from the data. However, it seems reasonable to state the 
subscores present little additional statistical information beyond that 
provided by the total score. However, these subscale scores may provide 
useful diagnostic information for planning educational programs. 

Results - Wicomico County 

Initial Tests 

During the 1974-75 school year, all 550 kindergarten youngsters 
in Wicomico County were evaluated using the MSTOI. From this group a 
total of 546 usable test forms were obtained. The mean of the total 
MSTOI score was 149.10 with a standard deviation of 19.95. Individual 
item means and subscore means are shown in Appendix A, Table I. 

In addition to the MSTOI, the Cognitive Skills Achievement Battery 
(CSAB) was administered to each kindergarten youngster. As a result of 
this testing, 546 usable CSAB test scores were obtained. The mean of 
the total score was 108.94 and the standard deviation was 17.55. These 
data are shown in detail in Appendix A, Table J. 

The youngsters were also rated using the Classroom Behavior 
Inventory . Observations were made at approximately the same time during 
the year. A total of 546 valid forms were available. For this initial 
sample, a mean of 28.89 was calculated with a standard deviation of 8.69. 
Subscale scores are shown in Table K in Appendix A. 
Follow-up Tests 

The criterion measure, the Pupil Rating Scale (PRS), was administered 
to all of the initial sample that could be located. A total of 387 
students were included in this follow-up sample. An analysis of the PRS 
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test data for these youngsters showed a total score mean of 75.88 with 
a standard deviation of 16.98. Additional information on the subscale 
scores is shown in Table L in Appendix A. 

The possibility of a systematic change in sample composition was 
investigated by comparing the mean scores of MSTOI initial sample with 
the mean score of the follow-up sample. On the initial sample, the 
MSTOI mean was 149.10 with a standard deviation of 19.95. On the 
follow-up sample, the MSTOI mean was 151.06 with a standard deviation 
of 18.33. Subscale scores for the follow-up sample are shown in Table 
M. A t-test was performed and the results showed no significant 
differences at the .05 level. Since there was no significant difference 
between the samples, results from the follow-up sample can be general- 
ized to the initial sample. 
Concurrent Validity 

The concurrent validity of the MSTOI for assessing Wicomico 
County students was established by determining the relationship 
between students' MSTOI ratings and their ratings on the other initial 
instruments. The results are summarized in tables 19-21. Table 19 
indicates the distribution of CSAB and MSTOI scores when they are 
classified as either above or below the cutoff point. The results from 
the X2 analysis indicated a statistically significant degree of associa- 
tion between MSTOI and CSAB ratings. This apparent relationship is showi 
in the fact that no student who was placed below the cutoff point on the 
MSTOI scored above the CSAB total score cutoff point. 



Table 19 

Distribution Of Wicomico County Follow-up Sample 
Classified By MSTOI Performance And CSAB Total Scores 



MSTOI Total Score 



CSAB Total Score 



128 or Greater Less than 128 



Row Total 



139 or Greater 



Less than 139 



Column Total 



N 

Row % 
Col. % 

N 

Row % 
Col. % 

N 

% 



48 
15 
100 

0 
0 
0 

48 
13 



264 
85 
79 

72 

100 
21 

336 
87 



312 

81% 

72 

19% 

384 
100 



Chi -square = 11.29 
Phi = .18 



P < .01 



Table 20 indicates the distribution of MSTOI and CBI scores that 
have been classified according to the cutoff points for these tests. 
The statistically significant value computed indicated a significant 
relationship between MSTOI scores and CBI scores i.e. concurrent validity. 
It is interesting to note that over one. half of the students were classi- 
fied below the cutoff point for the CBI whereas only 19% of the students 
were found to be below the MSTOI cutoff score. 



3 ; 
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TABLE 20 

Distribution Of Wicomico County Follow-up Sample 
Classified By MSTOI Performance And CBI Total Scores 



MSTOI Total Score 



CBI Total Score 





32 or Greater 


Less than 32 


Row Total 


N 


169 


142 


311 


Row % 


53 


46 


81% 


Col. % 


97 


68 


N 


6 


68 


74 


Row % 


8 


92 


19% 


Col. % 


3 


32 


N 


175 


210 


385 


% 


46 


54 


. 100 



139 or Greater 



Less than 139 



Column Total 



Chi-square = 49.68 
Phi = .37 



P ^ .01 



A second procedure for determining the concurrent validity of the 
MSTOI was employed. The reasons for testing the relationship between 
MSTOI performance and the other initial testing instruments with the 
Pearson Product Moment Correlational technique as well as the were 
discussed in the section dealing with the Queen Anne's County results. 
In general, it might be appropriate to say that the technique 
provides results that have greater understandability and relevance to 
the classroom teacher. The Pearson Product Moment technique utilizes 
the data more completely and provides results that are more precise and 
amenable to further statistical analysis. All of the correlation 
coefficients indicated a significant .01 level relationship between 
MSTOI ratings and the other test performances. In this analysis, the 
correlations of the MSTOI with the CSAB and CBI were computed. These 
data are shown in Table 21. ,' 
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TABLE 21 

Pearson Product Moment Correlation Coefficients Between MSTOI Scores 
And The CSAB And The CBI For The Wicomico County Follow-up Sample 



MSTOI Subscales 


CSAB Total 

IN OOH 


CBI Total 

IN- JoD 


D c \/f* h nmn "f* n v» 

rbyuiiuiiiu Lur 


fid 


• DC. 


Language 


.74 


.62 


Sensory/Perception 


.74 


.61 


Cognition 


.70 


.54 


Affect/Motivation 


.56 


.65 


MSTOI Total 


.75 


.67 



Note: All correlations significant at the .01 level . 



Additional information concerning the relationship of the HSTOI 
with the CSAB and CBI subscales is shown in Appendix A, Tables N and 0. 
Predictive Validity 

The predictive validity of the initial assessment instruments, MSTOI, 
CSAB and CBI was determined by calculating the contingency coefficients 
and the Pearson Product Moment Correlation Coefficients with the criterion 
measure, the PRS. Tables 22-27 present the results of this aspect of the 
validity study. All statistical indicators of predictive validity were 
significant at the .01 level. Table 22 indicates the distribution of 
MSTOI and PRS scores classified as either above or below the cutoff score 
for the respective test. The calculated indicated that there is a 
statistically significant degree of association between performances on 
the two tests. 
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TABLE 22 



Distribution Of Wicomico County Follow-up Sample 
Classified By MSTOI Performance And PRS Total Score 



MSTOI Total Score 



139 or Greater 



Less than 139 



Column Total 



PRS Total Score 



N 

Row % 
Col. % 

N 

Row % 
Col. % 

N 



62 or Greater 



Less than 62 



277 
89 
87 

40 
54 
13 

317 
82 



35 
11 
51 

34 
46 
49 

69 
18 



Row Total 



312 
81% 
74 
19% 

386 

100 



Chi -square = 46.80 
Phi = .36 



P ^ .01 



Table 23 summarizes the results of analyzing the relationship 
between the various MSTOI and PRS scales with the Pearson Product 
Moment Correlation technique. The predictive validity of the MSTOI 
total score when using the PRS total score as the criterion is .52. 
Inspection of the correlation coefficients indicating the relationships 
among the various subscale scores reveals that the MSTOI language sub- 
scale possess predictive validity coefficients of approximately the same 
magnitude as the MSTOI total score. The PRS Motor Coordination subscale 
is notable in that its coefficients appear to be of a lower order than the 
other subscales. 
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TABLE 23 

Pearson Product Moment Correlation Coefficients Between The 
MSTOI And PRS For The Wicomico County Follow-up Sample (N=386) 



MSTOI Subscores PRS Subscores 





Auditory 
Comprehension 


Spoken 
Language 


Orientation 


Motor 

Coordination 


Personal/Social 
Behavior 


Verbal 


Nonverbal 


PRS Total 


Psychomotor 


.42 


.43 


.40 


.24 


.46 


.46 


.45 


.47 


Language 


.48 


.48 


.42 


.27 


.45 


.51 


.45 


.50 


Sensory/ 


















Perception 


.44 


.47 


.41 


.28 


.44 


.48 


.43 


.48 


Cognition 


.38 


.41 


.33 


.18 


.37 


.43 


.35 


.40 


Affect/ 


















Motivation 


.42 


.41 


.42 


.23 


.49 


.44 


.47 


.48 


MSTOI Total 


.48 


.49 


.43 


.27 


.40 


.52 


.48 


.52 



Note: All correlations significant at the .01 level. 



The predictive validity of CSAB was determined by developing a 
contingency table and by determining the correlation with the criterion 
measure. Score distributions of the CSAB and PRS are shown in Table 24. 
Table 25 presents the results of Pearson Product Moment Correlation 
analysis. All calculated values were significant at the .01 level. 
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TABLE 24 



Distribution Of Wicomico County Follow-up Sample 
Classified By CSAB Performance And PRS Scores 



CSAB Total Scores 




PRS Total 

DC or area Lci 


Score 

LcS S tllail QC 


r\UW 1 u La 1 


128 or Greater 


N 


47 


1 


48 




Row % 


98 


2 






Col. % 


15 


1 


^z% 


Less than 128 


N 


268 


69 


337 




Row % 


80 


20 






Col. % 


85 


99 


S7% 


Column Total 


N 


315 


70 


385 




% 


82 


18 


100 



Chi -square =8.36 P -01 

Phi = .16 



The predictive validity of the CSAB with the PRS as the criterion was 
indicated by the correlation coefficient of .50. Correlation coefficients 
indicating the relationships among the competency areas and the PRS sub- 
totals are shown in Table 25. All computed values were significant. 



TABLE 25 

Pearson Product Moment Correlation Coefficients Between CSAB 
Scores And PIAT For Wicomico County Follow-up Sample (N=385) 



CSAB PRS Subscores 

Competency _ 

Areas Verbal Nonverbal Tota l 

Orientation .51 .44 .49 

Coordination .33 .34 .35 

Discrimination .46 .39 .44 

Memory .49 .44 .48 

Comprehension/ 

Concept .52 .44 .50 

CSAB Total .56 .48 .54 



Note: All correlations significant at the .01 
level . 
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The predictive validity of the initial assessment measure, the 
CBI, with the PRS as a criterion measure is summarized in Tables 26 
and 27. All calculated values were significant at the .01 level. 
Distribution of the CBI scores and the PRS scores according to their 
cutoff points are documented in Table 26. 



TABLE 26 

Distribution Of Wicomico County Follow-up Sample 
Classified By CBI Performance And PRS Total Scores 



CBI Total Scores 




PRS Total 
62 or Greater 


Scores 
Less than 62 


Row Total 


32 or Greater 


N 


163 


12 


175 




Row % 


93 


7 






Col. % 


52 


17 


45% 


Less than 32 


N 


153 


58 


211 




Row % 


73 


27 






Col. % 


48 


83 


55% 


Column Total 


N 


316 


70 


386 




% 


82 


18 


100 



Chi-square = 26.05 P ^ .01 

Phi = .27 



The predictive validity of the CBI was established by determining 
its correlational relationship with the criterion PRS scores. The 
correlations were significant at the .01 level. These data are shown in 
Table 27. 

TABLE 27 

Pearson Product Moment Correlation Coefficients Between CBI Total 
Scores And PRS Scores For Wicomico County Follow-up Sample (N=386) 

PRS Subscales 

Verbal Nonverbal Total 

CBI Total .42 .43 .45 

Note: All correlations significant at the 
.01 level. 
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Item Analysis 

In order to assess the predictive validity of the MSTOI, an item 
analysis was performed. A correlation coefficient was computed 
indicating the relationship of each MSTOI item and MSTOI subscale 
score with the MSTOI total score and the PRS criterion measure. All 
coefficients are significant at the .01 level. These data are shown 
in Table 28. 
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TABLE 28 



MSTOI Item Analysis 
Correlations Of Each Item With Total MSTOI Score And PRS 
Criterion Measure For Wicomico County Follow-up Sample (N=386) 

MSTOI Item MSIUI Subscaies 



O C C (O 





Psychomot( 


Language 


Sensory/ 
Perceptioi 


Cognition 


Affect/ 
Motivatioi 


MSTOI Tot 


PRS Total 


1 


.77 


.56 


.61 


.47 


.63 


.65 


.41 


2 


.58 


.57 


.59 


.50 


.60 


.63 


.41 


3 


.57 


.74 


.60 


.70 


.50 


.68 


.33 


4 


A "7 


. bb 


/I C 




/1 1 


. bo 


. CH 


5 


.58 


.41 


.41 


.36 


.55 


.54 


.26 


6 


.68 


.73 


.53 


.49 


.63 


.69 


.35 


7 


.49 


.42 


.31 


.29 


.66 


.51 


.24 


8 


.64 


.57 


.64 


.60 


.41 


.60 


.24 


9 


.76 


.62 


.64 


.56 


.77 


.73 


.42 


1 u 


.64 


.84 


.67 


.67 


56 


.75 


.37 


11 


.71 


.72 


.68 


.67 


.72 


.77 


.51 


12 


.48 


.62 


.55 


.64 


.32 


.57 


.34 


13 


.40 


.33 


.26 


.27 


.49 


.41 


.24 


14 






AT 
. 1 1 


fin 






?4 


15 


.52 


.34 


.30 


.32 


.72 


.50 


.29 


16 


.54 


.61 


.74 


.76 


.51 


.68 


.33 


17 


.54 


.54 


.62 


.71 


.38 


.60 


.26 


18 


.51 


.43 


.35 


.40 


.61 


.53 


.26 


19 


.48 


.49 


.53 


.68 


.39 


.57 


.30 




.62 


.56 


.55 


.46 


.68 


.65 


.37 


21 


.41 


.44 


.63 


.46 


.31 


.50 


.34 


22 


.43 


.31 


.30 


.29 


.66 


.47 


.25 


23 


.60 


.59 


.43 


.44 


.40 


.52 


.32 


24 


.71 


.73 


.69 


.63 


.75 


.78 


.50 


25 


.51 


.38 


.37 


.38 


.70 


.54 


.30 


26 


.46 


.50 


.59 


.67 


.42 


.57 


.33 


27 


.61 


.49 


.46 


.46 


.78 


.64 


.26 


28 


.60 


.83 


.68 


.66 


.56 


.74 


/n 

. r 1 


29 


.65 


.43 


.34 


.34 


.60 


.54 


.21 


30 


.50 


.57 


.74 


.59 


.39 


.60 


.34 


31 


.44 


.58 


.56 


.73 


.37 


.59 


.30 


32 


.52 


.59 


.59 


.72 


.47 


.63 


.32 


33 


.54 


.72 


.56 


.54 


.45 


.63 


.33 


34 


.54 


.59 


.72 


.53 


.47 


.63 


.31 


35 


.48 


• .58 


.65 


.43 


.44 


.59 


.42 


36 


.53 


.76 


.67 


.58 


.48 


.68 


.44 


Psychomotor 




.80 


.78 


.69 


.83 


.89 


.46 


Language 






.83 


.83 


.70 


.93 


.49 


Sensory/Percepti on 






.84 


.66 


.91 


.47 


Cognition 










.62 


.88 


.39 


Affect/Motivation 










.86 


.47 



MSTOI Total .50 
Note: All values significant at the .01 level. 
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Discussion 

Analysis of the Wicomico County data revealed that the MSTOI 
possesses both concurrent and predictive validity. A summary of these 
data are shown in Table 29 and a discussion of the data in the table 
follows. 

TABLE 29 

Summary Of Validity Indices For The 
MSTOI, CBI, And CSAB In Wicomico County 





X2 


r 


Concurrent Validity 


11.29 


.75 


MSTOI with CSAB 


MSTOI with CBI 


49.68 


.67 


Predictive Validity 






MSTOI with PRS 


46.80 


.52 


CSAB with PRS 


8.36 


.54 


CBI with PRS 


26.05- 


.45 



Note: All values significant at the 
.01 level. 



In the discussion of the results, the Pearson Product Moment 
Correlational analysis will be presented first and will be followed 
by a discussion of the x2 analysis of data. The MSTOI concurrent 
validity correlation coefficients of .67 and .75 indicate a strong 
relationship with the initial assessment instruments. Of major 
concern in this study, however, was the predictive validity of the 
initial assessment instruments, the MSTOI, CSAB and CBI. For 
predictive validity, the correlation coefficients ranged from .45 to 
.54. In order to determine whether one test was a significantly better 
predictor, a t-test was done on the dependent correlations. No signif- 
icant differences were found in the dependent correlations. It was 
concluded that all three initial assessment instruments were equally 
good predictors when the PRS was used as the criterion instrument. 
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Examination of the chi -square data showed that the MSTOI had 
concurrent validity at .01 level of significance. The predictive 
validity as determined by showed that all of the initial assessment 
instruments had significant predictive validity at the .01 level. As 
shown in Table 29, however, there was a wide difference in actual 
values obtained. In part, this range can be explained by examining 
the cutoff scores for various instruments and the resulting proportion 
of the sample identified as falling below the cutoff point. An 
examination of the data for this county showed that when the MSTOI 
was used, approximately 19% of the sample scored below the MSTOI cutoff 
point. In contrast, the percentage of students scoring below the 
cutoff point on the CSAB and the CBI was 88% and 54% respectively. 

Earlier data had indicated that the percentage of students rated 
below the 139 cutoff score would be approximately 40%. The wide 
discrepancy between an anticipated 40% and the observed 19% may be 
attributed to several factors. First, the sampie may well represent 
a group of students who are above average ability when compared to 
the rest of the state. A second explanation is that since all MSTOI 
tests were not administered in the first eight weeks of the school, 
students progressed sufficiently during the school year to be evaluated 
as being above the cutoff score. Third, teacher bias may have developed 
where teachers tend to evaluate students above the cutoff score. Such 
an evaluation bias would reduce the number of EMP*s written for students. 
The extent or existence of these practices cannot be determined from 
the data available. 

It may be hypothesized that the high percentage of students scoring 
below the cutoff point on the CSAB can be attributed to the high cutoff 
score used with this test. Although the number of students falling 
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below the CBI cutoff point was 54%, these data may indicate that the 
sample is a group that has less ability than the norm population. 

The results of the item analysis may be summarized as follows: 
1), all individual items were related significantly with the total 
MSTOI score and with the PRS criterion measure, 2) all of the sub- 
scales had validity coefficients significant at .01 level with the 
criterion measure total score, 3) all of the subscores were inter- 
correlated significantly. The interrelationships among the five 
subscales ranged from .62 to .84. 

Results - Cecil County 

Initial Tests 

In Cecil County, the MSTOI was used to evaluate all of the 
kindergarten youngsters. A total of 435 usable test forms were 
obtained. The mean for this population was 145.00 with a standard 
deviation of 23.19. Individual item means and subscale means are 
shown in Table P, Appendix A. 

On the basis of the total MSTOI scores, 100 youngsters were 
selected for additional testing. The youngsters with the fifty 
highest MSTOI scores and the youngsters with the fifty lowest scores 
were chosen for additional testing. Where possible, the Meeting Street 
School Screening Test (MSSST) and the Self Control Behavior Inventory 
(SCBI) were administered to these children. From this group, 88 
usable pairs of MSSST and SCBI scores were obtained. This group of 
88 was designated as the initial sample. Since the selection of the 
initial sample was not random, it was necessary to know whether the 
initial sample significantly differed from population on the MSTOI 
scores. The mean of the initial sample for the MSTOI was 141.18 with 
a standard deviation of 33.00. A t-test was conducted to assess the 
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significance of the difference in the means between the population and 
the initial sample. Results of the t-test indicated the means of these 
samples differed significantly at the .01 level of confidence. There- 
fore it was concluded that all statistical analysis by necessity would 
be limited to the initial sample and could not be generalized to the 
population. 

For the initial sample, the mean total score on the MSSST was 48.43 
with a standard deviation of 20.68. The MSSST subscales are shown in 
Table Q. On the SCBI, the mean of the total score was 13.41 with a 
standard deviation of 7.86. 
Follow-up Study 

In any longitudinal study, a sample loss can be predicted. In 
this case, the original 88 usable scores was reduced to a 79 follow-up 
sample. The criterion measures, the PIAT and PRS, were administered 
to these 79 youngsters. Criterion measures were administered in the 
spring of 1977. Results showed a PIAT total score mean of 160.84 
with a standard deviation of 58.36. For the follow-up sample, the 
total score on the PRS for the total score was 79.38 with a standard 
deviation of 22.40. Subscale scores on the criterion measures are 
shown in Tables R and S in Appendix A. 

Prior to proceeding with any additional data analysis, the MSTOI 
total score mean for the initial sample was compared with the MSTOI 
total score mean for the follow-up sample. The mean for the initial 
sample was 141.18 while the mean for the follow-up sample was 136.01. 
Data for the MSTOI follow-up are shown in detail in Table T. No 
significant difference in the means was detected with a t-test. From 
this result, it was inferred that all future calculations could be 
made using the data from the follow-up sample. All conclusions 
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reached could be generalized to the initial sample but not to the 
population. Follow-up data for the MSSST is shown in Table U. The 
SCBI follow-up sample mean is 11.03 with a standard deviation of 8.44. 
Concurrent Validity 

An analysis was done to determine the concurrent validity of MSTOI. 
First, the MSTOI total score distribution was determined with the score 
distributions of MSSST. These results are shown in Table 30. The 
relationship of the score distributions was significant at the .01 
level . 



TABLE 30 

Distribution Of Cecil County Follow-up Sample 
Classified By MSTOI Performance And MSSST Total Scores 



MSTOI Total Score 



139 or Greater 



Less than 139 



Column Total 



N 

Row % 
Col. % 

N 

Row % 
Col. % 

N 

% 



MSSST Total Scores 



58 or Greater Less than 58 



25 
69 
96 

1 
3 
4 

26 
38 



n 

31 
26 

32 
97 
74 

43 
62 



Row Total 



36 

52% 

33 

m 

69 
100 



Chi -square = "«i9.57 
Phi = .68 



■p ^ .01 



The concurrent validity of the MSTOI also was determined by 
examining the relationship between MSTOI total score distribution and 
SCBI total score distribution. These data are shown in Table 31. The 
relationship between the scores on the two rating instruments was signif- 
icant at the .01 level . 
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TABLE 31 

Distribution Of Cecil County Follow-up Sample 
Classified By MSTOI Performance And SCBI Total Scores 



MSTOI Total Score 




SCBI Total 

1 \} Ui Ui cu Lci 


Score 

LCd d LI lu 1 1 1 


Pnw Total 


139 or Greater 


N 


31 


5 


36 




Row % 


86 


14 






Col. % 


97 


14 


53% 


Less than 139 


N 


1 


31 


32 




Row % 


3 


97 






Col. % 


3 


86 


47% 


Column Total 


N 


32 


36 


68 




% 


47 


53 


100 



Chi-square = 43.56 P .01 
Phi = .83 



The concurrent validity of the MSTOI was determined by computing 

I 

the Pearson Product Moment Correlation Coefficients for the MSTOI with 
the two initial assessment instruments, the MSSST and the SCBI. MSTOI 
total score with the total score on the MSSST showed a correlation of 
.68. The Pearson Product Moment Correlation Coefficient for the MSTOI 
total score with the SCBI total score was .76. Correlations for the 
various subscores are shown in Table 32'. The relationship between the 
two teacher rating instruments was consistently higher than the relation- 
ship between the MSTOI and the MSSST. All relationships were significant 
at .01 level . 
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TABLE 32 



Pearson Product Moment Correlation Coefficients Between MSTOI Scores 
And MSSST And SCBI Scores For Cecil County Follow-up Sample (N=68) 



RSTOI 
Subscales 



MSSST Subscores 



Motor 



Visual 
Perceptual 



MSSST 
Total 



SCBI 
Total 



Psychomotor 


.63 


.66 


.71 


.68 


.75 


Language 


.63 


.64 


.69 


.68 


.74 


Sensory/ 
Perception 


.64 


.66 


.69 


.69 


.73 


Cognition 


.62 


.64 


.66 


.66 


.71 


Affect/ 
Motivation 


.61 


.63 


.65 


.65 


.76 


MSTOI Total 


.63 


.66 


.69 


.68 


.76 



Note: Sll correlations significant at .01 level. 



Predictive Validity 

The predictive validity of the three Initial assessment instruments 
was determined by examining the score distribution of the initial assess- 
ment instruments and of tf. criterion measures Peabody Initial Achievement 
Test (PIAT) and Pupil Rating Scale (PRS). Predictive validity also was 
determined by computing the Pearson Product Moment Correlation Coefficients 
between the three initial assessment instruments and the two criterion 
measures. 

The distribution of scores for the MSTOI initial assessment measure 
and the PIAT criterion measure are shown in Table 33. It should be 
noted that, relationship between scores was significant at the .01 level. 
No students scoring above 139 on the MSTOI scored below the PIAT cutoff 
point. 
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TABLE 33 

Distribution Of Cecil County Follow-up Sample 
Classified By MSTOI Performance And PIAT Total Scores 



MSTOI Total Score 




PIAT Total 
102 or Greater 


Score 

Less than 102 


Row Total 


139 or Greater 


N 


40 


0 


40 




Row % 


100 


0 






Col . t 


63 


0 


51% 


Less than 139 


N 


24 


15 


39 




Row % 


62 


38 






Col. % 


37 


100 


49% 


Column Total 


N 


64 


15 


79 




% 


81 


19 


100 


Chi-square = 16.57 P ^ .01 
Phi = .49 

The Pearson Product Moment 'Correlation between the MSTOI total scores 
and the PIAT total scores was .85. Correlation for the subscores are 
shown in Table 34. All correlations were significant at the .01 level. 



TABLE 34 

Pearson Product Moment Correlation Coefficients Between MSTOI 
And PIAT Scores For Cecil County Follow-up Sample (N=79) 



MSTOI Subscales PIAT SubscaUs 



o 

o o to 

e cc CCD-'- fOEj— 

0) CU (DO ^ ^ 



o 
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Psychomotor 


.77 


.85 


.86 


.83 


.70 .86 


Language 


.77 


.84 


.85 


.83 


.71 .85 


Sensory/Perception 


.80 


.84 


.87 


.84 


.71 .87 


Cognition 


.72 


.77 


.79 


.77 


.64 .79 


Affect/Motivation 


.77 


.82 


.82 


.76 


.70 .83 


MSTOI Total 


.77 


.83 


.85 


.81 


.71 .85 
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The distribution of the Cecil County follow-up sample classified 
by MSTOI total scores and the criterion PRS are shown in Table 35. The 
relationship between the score distributions is significant at the .01 
level . 



TABLE 35 

Distribution Of Cecil County Follow-up Sample 
Classified By MSTOI Performance And PRS Total Scores 



MSTOI Total Score 




PRS Total 
62 or Greater 


Score 

Less than 62 


Row Total 


139 or Greater 


N 


39 


1 


40 




Row % 


98 


2 






Col. % 


71 


4 


51% 


Less than 139 


N 


16 


23 


39 




Row % 


41 


59 


49% 




Col. % 


29 


96 


Column Total 


N 


55 


24 


79 


% 


70 


30 


100 



Chi-square = 27.17 P ^ .01 
Phi = .61 



An analysis of the relationship between the MSTOI total scores and 
the PRS total showed a correlation coefficient of .81. The relationships 
between the subscales are shown in Table 36. The subscale scores corre- 
late almost as highly as the total scores. The cognition subscale on 
the MSTOI shows the lowest degree of correlation with the PRS. 
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TABLE 36 

Pearson Product Moment Correlation Coefficients Between MSTOI 
And PRS Scores For The Cecil County Follow-up Sample (N=79) 

MStOl Subsca'les PRS Subscales 



• 


Auditory 
Comprehension 


Spoken 
Language 


Orientation 


Motor 

Coordination 


Personal/Soci 
Behavior 


Verbal 


Nonverbal 


PRS Total 


Psychomotor 


.76 


.80 


.79 


.75 


.78 


.80 


.83 


.84 


Language 


.76 


.80 


.75 


.70 


.75 


.79 


.79 


.82 


Sensory/Perception 


.75 


.77 


.75 


.69 


.76 


.78 


.80 


.81 


Cognition 


.67 


.73 


.68 


.66 


.69 


.71 


.73 


.74 


Affect/Motivation 


.76 


.77 


.75 


.71 


.79 


.78 


.81 


.83 


MSTOI Total 


.75 


.77 


.74 


.71 


.75 


.77 


.79 


.81 



Note: All correlations significant at the .01 level. 



The efficacy of the MSTOI as a predictor of academic success is 
indicated by the distribution of grade placement and MSTOI total scores. 
These data are shown in Table 37. Only one student scoring above the 
MSTOI cutoff score did not make normal school progress. The relation- 
ship between the two distributions was significant at the .01 level. 
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TABLE 37 

Distribution Of Cecil County Follow-up Sample 
Classified By MSTOI Performance And Grade Placement 



MSTOI score 






Grade Placement 
First Second 


Row Total 


139 or Greater 


N 




1 


39 


4U 


Row 


% 


3 


97 


51% 




Col 


. % 


4 


71 


Less than 139 


N 




22 


16 


38 




Row 


% 


58 


42 






Col 


. % 


96 


29 


49% 


Column Total 


N 




23 


55 


78 


% 




29 


71 


100 



Note: One grade placement missing. 
Chi-square ^ 26.16 P .01 

Phi = .61 

The second initial assessment administered was the Meeting Street 
School Screening Test (MSSST) . The distribution of the MSSST total 
scores and the PIAT criterion measure are shown in Table 38. The 
relationship between the score distributions was significant at the 
.01 level. No student scoring above the MSSST cutoff fell below the 
PIAT cutoff score. 



TABLE 38 

Distribution Of Cecil County Follow-up Sample 
Classified By MSSST Performance And PIAT Total Score 



MSSST Total Score 



PIAT Total Score 
102 or Greater Less than 102 Row Total 



58 or Greater 

Less than 58 
Column Total 



N 

Row % 
Col. % 

N 

Row % 
Col. % 

N 

% 



26 
100 
46 

31 
72 
54 

57 
83 



0 
0 
0 

12 
28 
100 

12 
17 



26 

38% 

43 

62% 

69 
100 
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The predictive validity of the MSSST was determined by examining 
the relationship between the MSSST and the PIAT. A correlation of .82 
was found for the total scores. The relationships between the subscales 
are shown in Table 39. All correlations are significant at the .01 
level . 

TABLE 39 

Pearson Product Moment Correlation Coefficients Between MSSST 
And PIAT Total Scores For Cecil County Follow-up Sample (N=69) 



MSSST Subscales PIAT Subscales 



c 
o 

to C -r- C 

O O </> O r— 

•r- -r- C T- <0 

rO cn-r- cn-C c r— (O p 

-c "oo -oa. r- cup H 

rtj (U(U (UO Q- cue » 

s: cxioel O CO CD »-H Q- 



Motor 
Patterning 


.70 


.66 


.70 


.64 


• 

.60 


.71 


Visual 

Perceptual /Motor 


.76 


.72 


.79 


.70 


.57 


.76 


Language 


.80 


.80 


.81 


.75 


.73 


.84 


Total 


.80 


.77 


.81 


.74 


.67 


.82 



The predictive validity of the MSSST was determined by comparing 
the score distribution of the MSSST with the score distributions on 
the criterion measure, the PRS. This distribution is shown in Table 40, 
The relationship between the score aistributions is significant at the 
.01 level. Every student exceeding rhe MSSST cutoff score also scored 
above the PRS cutoff point. 
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TABLE 40 



Distribution Of Cecil County Follow-up Sample 
Classified By MSSST Performance And PRS Total Scores 



MSSST Total Score 




PRS Total 
62 or Greater 


Score 

Less than 62 


Row Total 




N 


26 


0 


26 




Row % 


1 A A 


0 






Col. % 


53 


0 


38% 


Less than 58 


N 


23 


20 


43 




Row % 


54 


46 






Col. % 


47 


100 


62% 


Column Total 


N 


49 


20 


79 




% 


71 


29 


100 



Chi-square = 14.84 P .01 

Phi = .50 



The predictive validity of the MSSST also was assessed by determining 
the correlation between the MSSST and the PRS. The correlation coefficient 
between the total scores was .81. Subscore correlations are shown in Table 
41. All correlations are significant at the .01 level. 

TABLE 41 



Pearson Product Moment Correlation Coefficients Between MSSST 
And PRS Scores For Cecil County Follow-up Sample (N=79) 



MSSST Subscales 






PRS 


Subscales 








pRS Total 


Auditory 
Comprehension 


Spoken 
Language 


Orientation 


Motor 

Coordination 


Personal/Social 
Behavior 


Verbal 


Nonverbal 


Motor 


















Patterning 


.64 


.69 


.67 


.58 


.70 


.68 


.71 


.72 


Visual/ 


















Perceptual Motor 


.74 


.71 


.79 


.64 


.77 


.74 


.80 


.79 


Language 


.76 


.77 


.69 


.61 


.76 


.78 


.75 


.79 


Total 


.75 


.77 


.75 


.64 


.79 


.77 


.80 


.81 



Note: All correlations are significant at the-trOl level. 
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Assessment of the predictive validity of the SCBI was made by 
examining the score distribution of the SCBI and the PIAT. These data 
are shown in Table 42. Unlike the other relationships in Cecil County 
sample, this relationship was not significant at the .01 level, but 
was significant at the .05 level. 

TABLE 42 

Distribution Of Cecil County Follow-up Sample 
Classified By SCBI Performance And PIAT Total Scores 

SCBI Total Score PIAT Total Scores 



102 or Greater Less than 102 Row Total 



15 or Greater N 




31 


1 


32 


Row % 




97 


3 




Col. % 




54 


9 


47% 


Less than 15 N 




26 


10 


36 


Row % 




72 


28 




Col. % 




46 


91 


53% 


Column Total N 




57 


11 


68 


% 




84 


16 


100 


Chi-square =5.88 P 


^ . 


05 






Phi = .33 










The correlation between 


the 


SCBI total score 


and PIAT 


scores was 


determined. Correlation of 


total 


scores was .61 . 


Subscore correlations 



are listed in Table 43. Total score and subscale scores are significant 
at the .01 level. 
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TABLE 43 



Pearson Product Moment Correlation Coefficients Between SCBI 
Scores And PIAT Scores For Cecil County Follow-up Sample (N=68) 

PIAT Subscore 



to 
a 



0) 



•r- CD 

•O O 

0) 0) 



c 

o 

•r- 

t/) 
C 
0) 

•r- S- 

•o a. 

(U o 
Od o 



O r— 
•r- ro 



0) 

a. 

CO 



ro E 
(U o 
(U C 



I— I 



SCBI Total .58 



.62 



.63 



.57 .51 



.61 



Note: All correlations are significant at the 
.01 level. 



The distribution of SCBI scores with the PRS scores was examined 
in order to assess predictive validity of the SCBI. These score 
distributions are shown in Table 44. Only one student exceeding SCBI 
cutoff score in kindergarten failed to exceed the PRS cutoff score 
in the second grade. 



TABLE 44 

Distribution Of Cecil County Follow-up Sample 
Classified By SCBI Performance And PRS Scores 



SCBI Total Score 



PRS Total 







62 or Greater 


Less than 62 


Row Total 


15 or Greater 


N 


31 


1 


32 




Row % 


97 


3 






Col. % 


63 


5 


47% 


Less than 15 


N 


18 


18 


36 




Row % 


50 


50 






Col . % 


37 


95 


53% 


Column Total 


N 


49 


19 


68 




% 


72 


28 


100 



Chi-square = 16.23 
Phi = .46 



P < .01 
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The predictive validity of the SCBI was determined by examining 
the strength of the relationship between the SCBI and the PRS. A 
correlation of .70 was found between the total scores. Subscale 
correlations are shown in Table 45. All correlations ranging from 
.63 to .74 are significant at the .01 level. 

TABLE 45 

Pearson Product Moment Correlation Coefficients Between SCBI Scores 
And PRS Scores For The Cecil County Follow-up Sample (N=68) 



PRS Subscores 
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c 
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o o 
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O 


ca:o 


CO -I 






CL CO 
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»- 


.74 


.74 


.73 


.69 


.78 


.63 


.72 


.70 



SCBI Total Score .74 
Note: All correlations significant at the .01 level. 



Item Analysis 

In order to assess the predictive validity of the MSTOI, an item 
analysis was performed. Each MSTOI item and subscore were compared to 
the MSTOI total score. Each item and subscore was compared to the 
total score of the criterion measures, the PI AT and the PRS. The 
relatively high correlations of the MSTOI subscores with the MSTOI total 
score should be noted. 
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TABLE 46 



MSTOI Item Analysis Correlations Of Each Item With MSTOI Subscales 
And MSTOI Total Score For Cecil County Follow-up Sample (N=79) 



MSTOI Item 



MSTOI SuDScale 



ERIC 





Psychomotor 


Language 


Sensory/ 
Perception 


Cognition 


Affect/ 
Motivation 


MSTOI Total 


1 


.82 


.76 


.80 


.67 


.80 


.78 


2 


.82 


.82 


.83 


.74 


.86 


.83 


3 


.90 


.93 


.90 


.89 


.87 


.92 


4 


.61 


.68 


.66 


.77 


.bb 


.by 


5 


71 






.66 


C7 

m \J / 


71 

. / 1 


6 




90 


.88 


.84 


.84 


.90 


7 


7*^ 

mis) 




69 


.68 


.77 


.74 


8 


. / U 




75 


.76 


.63 


.73 


9 


Qn 


. (J «^ 


85 


.84 


.91 


.89 


10 


. oo 




91 


.92 


.86 


.94 


n 


.87 


.87 


.87 


.85 


.89 


.89 


12 


.80 


.90 


.86 


.90 


.72 


.88 


13 


.59 


.61 


.63 


.51 


.59 


.61 


14 


.71 


.79 


.11 


.85 


.66 


.80 


15 


.44 


.40 




AT 


. D 1 


d7 


16 


.88 


.92 


.9b 




.o'f 




17 


r- -J 

.67 


"7 C 

.75 


70 
.IL 


• OD 


• 0 1 


. / D 


18 


.75 


"7 C 

. /5 


. // 


71 
. / 1 


7Q 


7Q 


19 


.68 


.69 


71 


77 


. DO 


71 

. / 1 


20 


"7 O 

. /3 


CO 

. bo 


7n 
. /u 


• DO 


PI 
. O 1 


74 


C i 


.74 


.75 


,80 


.69 


.77 


.77 


99 


.54 


.51 


.50 


.51 


.68 


.56 


9'\ 


.83 


.85 


.79 


.79 


.68 


.81 




.84 


.85 


.86 


.81 


.87 


.87 




.60 


.54 


.54 


.57 


.71 


.61 


26 


.75 


.83 


.84 


.85 


.71 


.84 


27 


.56 


.54 


.50 


.51 


.71 


.59 


28 


.89 


.95 


.92 


.91 


.83 


.93 


29 


.74 


.62 


.62 


.56 


.67 


.66 


30 


.84 


.87 


.91 


.87 


.77 


.88 


31 


.76 


.84 


.84 


.89 


.72 


.85 


32 


.81 


.87 


.86 


.91 


.74 


.8b 


33 


.79 


.89 


.85 


.86 


.69 


.85 


34 


.82 


.87 


.88 


,86 


.75 


.88 


35 


.83 


.89 


.91 


.87 


.79 


.89 


36 


.88 


.93 


.93 


.87 


.84 


.92 


Psychomotor 




.94 


.94 


.90 


.91 


.96 


Language 






.97 


.96 


.88 


.98 


Sensory/Percepti on 




.95 


.89 


.98 


Cognition 










.84 


.97 


Affect/Motivation 










.93 



Note: 'All correlations significant at the .01 
level. 59 
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TABLE 47 

MSTOI Item Analysis 
Correlations Of Each MSTOI Item And Subscale With The Total PIAT (N=79) 
And PRS (N=79) Criterion Measures For Cecil County Follow-up Sample 

MSTOI Item MSTOI Subscsle 





Psychomotor 


Language 


Sensory/ 
Perception 


Cognition 


Afl^ect/ 
Motivation 


PIAT Total 


PRS Total 


1 


.82 


.76 


.80 


.67 


.80 


.75 


.70 


c 


.82 


.82 


.83 


.74 


.86 


.78 


.76 


o 
J 


.90 


.93 


.90 


.89 


.87 


.81 


.79 


/I 


.61 


.68 


.66 


.77 


.55 


.53 


.47 


c 


71 


A7 
-0/ 


M 

. 


.DO 


fi7 
.0/ 


.30 


fil 

. D 1 


0 


• OD 


on 
.yu 


.OO 


. OH 


. OH 


R? 


7R 
• / 0 


7 






. 


. DO 


77 
. / / 


. oc 


•iR 

• JO 


p 
o 


7C 


7 Q 


. / 3 


. / D 


. 0 J 


fi7 

• D/ 


• D J 


9 


on 

.yu 






. ot 


. J 1 


. / D 


RD 
. ou 


10 


QQ 
. OO 




Ql 




. OD 


R9 


RD 
• ou 


11 


.87 


.87 


.87 


.85 


.89 


.75 


.78 


12 


.80 


.90 


.86 


.90 


.72 


.71 


.64 


13 


.59 


.61 


.63 


.51 


.59 


.60 


.50 


14 


.71 


.79 


.11 


.85 


.66 


.60 


. bb 


15 


44 


40 


.40 


.41 


.61 


.40 


.49 


16 


.88 


.92 


.95 


.93 


.84 


.81 


.78 


17 


67 


.76 


.12 


.86 


.61 


.55 


.50 


18 


7fi 


76 


.11 


.71 


.79 


.65 


.61 


19 




69 


.71 


.11 


.58 


.63 


.58 


20 




.68 


.70 


.65 


.81 


.65 


.64 


21 


.74 


.75 


.80 


.69 


.77 


.72 


.75 


22 


.54 


.51 


.50 


.51 


.68 


.45 


.51 


23 


.83 


.85 


.79 


.79 


.68 


.70 


.69 


24 


.84 


.85 


.86 


.81 


.87 


.79 


.80 


25 


.60 


.54 


.54 


.57 


.71 


.56 


.56 


26 


.75 


.83 


.84 


.85 


.71 


.64 


.60 


27 


.56 


.54 


.50 


.51 


.71 


.55 


.58 


28 


.89 


.95 


.92 


.91 


.83 


.82 


.79 


29 


.74 


.62 


.62 


.56 


.67 


.58 


.59 


30 


.84 


.87 


.91 


.87 


.77 


.79 


.73 


31 


.76 


.84 


.84 


.89 


.72 


.67 


.57 


32 


.81 


.87 


.86 


.91 


.74 


.69 


.67 


33 


.79 


.89 


.85 


.86 


.69 


.71 


.63 


34 


.82 


.87 


.88 


.86 


.75 


.73 


.67 


35 


.83 


.89 


.91 


.87 


.79 


.80 


.71 


36 


.88 


.93 


.93 


.87 


.84 


.82 


.76 


Psychomotor 




.94 


.94 


.90 


.91 


.86 


.84 


Language 






.97 


.96 


.88 


.85 


.82 


Sensory/Perception 






.95 


.89 


.87 


.81 


Cognition 










.84 


.79 


.74 


Affect/Motivation 










.83 


.83 


MSTOI Total 












.85 


.81 



Note: All correlations significant at the .01 level. 
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Discussion 

The procedures used to select the sample for study in Cecil County 
present a problem for those wishing to generalize beyond the immediate 
sample. Since only those at the extreme ends of the score distribution 
for the MSTOI were chosen, the sample scores were not normally distrib- 
uted j therefore, caution should be employed when interpreting results 
from various statistical analyses. The difference between MSTOI mean 
total scores for those chosen and the total school population (145.2 
vs. 140.65) should be noted. The resulting MSTOI score distribution 
is therefore not representative of the total school population. As 
far as may be inferred from the data, the MSTOI possesses both con- 
current and predictive validity. A summary of the validity indices for 
the MSTOI in Cecil County is presented in Table 48. 

TABLE 48 



Summary Of Validity Indices For The 
MSTOI, SCBI, And MSSST In Cecil County 





X2 


r 


Concurrent Validity 






MSTOI with MSSST 


29.57 


.68 


MSTOI with SCBI 


43.56 


.76 


Predictive Validity 






MSTOI with PRS 


27.17 


.81 


SCBI with PRS 


16.23 


.70 


MSSST with PRS 


14.84 


.81 


MSTOI with PI AT 


16.57 


.85 


SCBI with PIAT 


5.88* 


.61 


MSSST with PIAT 


6.95 


.82 



Note: All unstarred values signif- 



icant at the .01 level . 
♦Values significant at the 
.05 level . 
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Inspection of the preceding table reveals that in terms of 
predictive validity, the MSTOI compares favorably with the other 
screening instruments, SCBI and MSSST. An examination of the Pearson 
Product Moment Correlation Coefficients shows that the predictive 
validity of the MSTOI exceeds or equals the values calculated for the 
other initial assessment instruments. For this sample, the MSTOI 
teacher rating scale is as valid as the individually administered 
MSSST in predictive validity for both criterion measures. The pre- 
dictive validity of the MSTOI is larger than that of SCBI when the 
PIAT is used as a criterion measure. When the apparent difference 
between MSTOI coefficient of .85 and the SCPI coefficient of .61 was 
tested for significance, with the t-test, a significant difference 
was found. 

The predictive validity was assessed by constructing a contingency 
table and calculating the significance of the relationship between 
assessment instruments and the criterion measures. Five of the six 
calculations performed showed the value was significant at the .01 
level. The sixth value was significant at the .05 level. 

The results of the item and subscale item analysis in this county 
was similar to that reported for the other counties with the exception 
that here (perhaps due to the sample selection procedures) the corre- 
lations reported were higher than in the other counties. The results 
may be summarized as follows: 1) all of the subscales had predictive 
validity coefficients significant at the .01 level for both criteria 
(PIAT and PRS total score), 2) all items related significantly to the 
total MSTOI score and to PIAT and PRS total score, 3) the intercorre- 
lations among the various items and subscales were uniformly high. 
These results may be interpreted as sungested in discussions of the 
results from the other counties. 

ERIC 
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Additional evidence of the predictive validity of the MSTOI was 
observed when MSTOI scores were compared to student grade placement. 
MSTOI scores in the kindergarten were useful in predicting those 
students unlikely to make normal progress and were likely to be 
retained. 

The validity coefficients obtained in this study were higher 
than the ones calculated in the other two counties. In fact, the 
values are much greater than found with most tests over a two 
year period. These findings are in part a result of the sampling 
procedure employed in the study. Caution should be used when 
attempting to generalize these results to a more typical group. 

Results - Combined County 
Due to the various testing procedures employed and to the 
different assessment instruments used, it was not possible to combine 
the results of all counties. Where it could be justified, results 
were combined. The combined MSTOI asLjSsment is presented first 
since it gives statistical data about the instrument with a large 
number of students. These data are followed by the concurrent validity 
data for the MSTOI with the other initial assessment instruments. The 
predictive validity data for the MSTOI and the other initial assessment 
measures is presented. This is followed by the item analysis data for 
the combined sample. 

The MSTOI was the common initial assessment instrument utilized 
in all three counties. A total of 1285 MSTOI assessments were admin- 
istered. Analysis of the data showed a MSTOI mean score of 146.77 with 
standard deviation of 21.25. Individual item means and standard 
deviations for the combined sample are shown in Appendix A, Table V. 
As reported earlier, the MSTOI initial assessment means for the 
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individual counties were: Queen Anne's - 145.13, Wicomico - 149.10, 
and Cecil - 145.00. A one-way analysis of variance was performed to 
determine the significance of the difference existing among the means. 
The resulting F value of 5.72, which was significant at the .01 level, 
indicated the means were derived from different populations. 

Analysis of the follow-up sample (N=695) showed a mean MSTOI value 
of 147.61 with a standard deviation of 22.25. A t-test was utilized to 
test for a significant difference between the means of the initial 
sample and the follow-up sample. A significant difference between means 
was not found. A one-way analysis of variance was performed to test 
for significant differences among the three follow-up means of 145.80 
for Queen Anne's County, 151.06 for Wicomico, and 136.01 for Cecil 
County. The resulting F value of 16.88, which was significant at the 
-01 level, indicated the means were derived from different populations. 
Concurrent Validity 

It was possible to combine the data from Queen Anne's and Wicomico 
Counties in order to determine the overall concurrent validity coeffi- 
cients since the MSTOI, CSAB and CBI were administered in both counties. 
The data for the combined scores for the MSTOI/CSAB distribution are 
shown in Table 49. The relationship was significant at the .01 level. 
It is important to note that only 12% scored above the CSAB cutoff 
score with the greatest proportion of students classified as above the 
MSTOI cutoff score and below the CSAB cutoff score. 



o 
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TABLE 49 



Distribution Of The Combined Scores For Queen Anne's County And 
Wicomico County Classified By MSTOI Performance And CSAB Total Scores 



MSTOI Total Score 



CSAB Total Score 



128 pr Greater Less than 128 Row Total 



139 or Greater N 72 

Row % 15 
Col. % 100 


402 

75 


474 

78% 


Less than 139 N 0 

Row % 0 
Col. % 0 


137 
100 
25 


137 
22% 


Coiunin Total N 72 

% 12 


539 
88 


611 

100 


Chi-square = 22.15 P ^ .01 
Phi = .19 






The score distributions for the MSTOI and 


the CBI are shown in 


Table 50. The relationship between the score distributions is signif- 


icant at the .01 level with the majority of the students 


either above the 


cutoff score or below the cutoff score on both 


tests . 




TABLE 50 






Distribution Of The Combined Scores For Queen Anne'; 
Wicomico County Classified By MSTOI Performance And 


; County And 
CBI Total Scores 


MSTOI Total Score CBI lotal 


Score 




32 or Greater 


Less than 


32 Row Total 


139 or Greater N 278 

Row % 59 
Col. % 95 


197 
41 
61 


475 

77% 


Less than 139 N 13 

Row % 9 
Col. % 5 


127 
91 

39 


140 
23% 


Column Total N 291 

% 47 


324 

53 


615 
100 


Chi-square = 103.21 P -C .01 



Phi = .41 
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The Pearson Product Moment Correlation Coefficients were calculated 
to determine the degree of relationship between the MSTOI and the CSAB 
and CBI. The Pearson Product Moment Correlation Coefficient for the 
MSTOI with the CSAB was .65, for the MSTOI with the CBI was .68. Both 
of these values are significant at the .01 level. For convenience, the 
individual county values and combined values are summarized in Table 51. 



TA3LE 51 

Concurrent Validity Indices For The MSTOI With 
The CSAB and CBI for County And Combined Scores 



Sample + Comparisons 


X2 


r 


n 


Queen Anne's County 
MSTOI -CSAB 
MSTOI-CBI 


9.26 
56.52 


.72 
.73 


227 
230 


Wicomico County 
MSTOI-CSAB 


11 .29 
49.68 


.75 
.67 


384 
385 


Combined Queen Anne's 
and Wicomico 

MSTOI-CSAB 

MSTOI-CBI 


22.15 
103.21 


.73 
.68 


611 
615 



Notel All relationships significant at .01 
level . 



Predictive Validity 

By necessity, predictive validity for combined results was 
based on Queen Anne's County and Wicomico County data. Cecil County 
was not included in any combined results because of the sampling 
method used in that county. In Cecil County, the selection process 
of the highest and lowest performing students on the MSTOI was used 
to determine the follow-up sample. This selection procedure resulted 
in a nonnormal bimodal distribution of scores. Such a distribution, 
when combined with the other county scores, tends to produce 
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spuriously high correlations. 

In order to determine the predictive validity of the MSTOI with 
the PRS as criterion for the combined follow-up samples, a contingency 
table was developed. The distribution of the 616 scores for the MSTOI 
total scores and the PRS total score for the combined follow-up scores 
for Queen Anne's and Wicomico Counties are shown in Table 52. A 
relationship significant at the .01 level was found with 487 of the 
616 students similarly classified by both tests. 

TABLE 52 

Distribution Of The Combined Scores For Queen Anne's County And 
Wicomico County Classified By MSTOI Performance And PRS Total Scores 



KSTOI Total Score PRS Total Score 



62 or Greater Less than 62 Row Total 



139 or Greater N 

Row % 
Col. % 

Less than 139 N 

Row % 
Col. % 

Column Total N 

% 

Chi-square = 91.82 P < -01 

Phi = .39 

In order to determine the predictive validity of MSTOI for the 
combined samples, Pearson Product Moment Correlations were calculated 
for the MSTOI total scores with the PRS. The results are shown in 
Table 53. All of the Pearson Product Moment Correlation Coefficients 
indicating the strength of relationship between the MSTOI and the PRS 
scale and total scores were significant at the .01 level. 



423 53 476 

89 11 

85 45 77% 

76 64 140 

54 46 

15 55 23% 

499 117 616 

81 19 100 
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TABLE 53 

Pearson Product Moment Correlation Coefficients Between 
MSTOI Total Scores And For The Combined Queen Anne's 
County And Wicomico County Follow-up Sample (N=616) 

PRS Subscales 



MSTOI Total -49 
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.45 


.33 


.50 


.50 


.49 


.52 



Note: All values significant at the .01 level . 

Predictive validity of the CSAB and CBI was calculated. A 
summary of the individual county and combined county results is shown 
in Table 54. For the combined results, all predictive validity indices 
were significant at the .01 level. In ttrms of practical application, 
there appears to be little difference among the Pearson Product Moment 
Correlation Coefficients indicating the predictive validity of the 
various tests. 

TABLE 54 

Predictive Validity Indices For The MSTOI, CSAB, And CBI 
With The Criterion PRS For County And Combined Sample 



Sample + Comparison 


X2 


r 


n 


Queen Anne's County 








MSTOI -PRS 


31 .82 


.53 


230 


CSAB-PRS 


5.59* 


.58 


229 


CBI -PRS 


36.41 


.69 


231 


Wicomico County 








MSTOI -PRS 


46.80 


.52 


385 


CSAB-PRS 


8.36 


.54 


385 


CBI-PRS 


26.05 


.45 


386 


Combined Sample 








MSTOI -PRS 


91 .82 


.52 


616 


CSAB-PRS 


11.70 


.55 


614 


CBI-PRS 


61.21 


.47 


617 



O ' .01 level. 

ERJC *Significant at the .05 level, 



64 



As further indication of the predictive validity of the MSTOI, the 
relationship between MSTOI scores and student grade placement for the 
combined samples was examined. The distribution of scores for 615 students 
in Queen Anne's and Wicomico Counties is shown in Table 55. For this 
combined sample, a significant relationship at the .01 level was observed. 
When interpreting this table, the reader should be aware that 3% of the 
students were retained. 

TABLE 55 

D;-tribution Of The Combined Sample Of Queen Anne's 
County And Wicomico County Follow-up Sample 
Classified By MSTOI Scores And Grade Placement 



MSTOI Total Score Grade Placement 



Grade 1 Grade 2 Row Tot_al 



139 or Greater N 6 469 475 

Row % 1 99 

Col. % 29 79 m 



Less than 139 N 



15 125 140 



Row % 11 89 

Col. % 71 21 23% 

column Total N 21 594 615 



Chi-square = 26.49 P < .01 
Phi = .22 

Item Analysis 

The results of item analysis of the combined Queen Ann&'s County and 
Wicomico County MSTOI scores are presented in Table 56. All of the 
individual items are significantly correlated with the MSTOI subscale and 
total scores. Their values range from .39 to .80. In addition, the 
MSTOI subscale scores show a high positive relationship among themselves 
and with the MSTOI total score. 
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TABLE 56 

Correlation Of Each MSTOI Item With Total MSTOI And Subscale Scores 
For The Combined Queen Anne's County And Wicomico County Sample (N=616) 

MSToTTtem ' MSTOI Subscale 
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.46 
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.66 
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.47 
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Psychomotor 




.80 


.81 


.73 


.83 


.89 


Language 






.83 


.82 


.71 


.92 


Sensory/Perception 






.86 


.68 


.91 


Cognition 










.64 


.89 


Affect/Motivation 










.87 



Note: All values significant at the .01 level. 
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The correlations of the MSTOI items and subscales with the criterion 
measure, the PRS, are shown in Table 57. Individual MSTOI item corre- 
lations ranged from .16 to .54 with the PRS total score. MSTOI subscale 
correlation coefficients ranged from .43 to .54 with the PRS total score. 



TABLE 57 



MSTOI Item Analysis 
Correlation Of Each MSTOI Item For The Combined 
Queen Anne's County And Wicomico County Sample 
With The PRS Criterion Measure (N=615) 
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.50 


Language 
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Cognition 
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Affect/Motivation 


.48 


Total 


.54 



Note: All correlation coef- 
ficients significant 
at the .01 level . 
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Discussion 

The combined results were based on the Queen Anne's County and 
Wicomico County data. Cecil County data was not included in any of 
the combined results due to the sampling procedures employed in the 
county. In Cecil County the follow-up sample was composed of students 
that made the highest and lowest MSTOI scores. This selection process 
resulted in a nonnormal, bimodal distribution of scores. Such a 
distribution when combined with the other county scores tended to 
produce spuriously high relationships. 

By excluding the Cecil County data, the resulting combined score 
distribution did not have greater than chance values for kurtosis and 
skewness. As a result, the assumption was made that the combined ^ 
scores were normally distributed about the mean. Therefore, based on 
a normal curve distribution percentage of scores falling below the 139 
cutoff, would be 35% or 243 cases. The observed or actual percentage 
was 26% or 178 cases for this combined sample. Both the expected and 
the actual percentages were somewhat lower than the 40% figure estimated 
from earlier studies. Seve al explanations are possible for such a 
discrepancy. First, some of the tests were administered as late as 
January. Since the MSTOI measures aspects of developmental growth, 
maturation and learning would tend to reduce the proportion falling 
below the cutoff score. Another explanation to be considered is the 
possibility of teacher bias in making student evaluations. A sub- 
conscious systematic bias may operate to reduce the number of students 
identified as being special. Finally, the sample studies may not 
truly represent the state population and therefore the percentage of 
identified cases should not be compared with the state norm. 

The significantly higher MSTOI total for Wicomico County 
indicated that the overall level of kindergarten student performance 
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was better than for the other two counties. Although it was not 
possible to make CSAB comparisons among all three counties, it was 
possible to compare the CSAB scores for Wicomico and Queen Anne's 
Counties. Unlike the MSTOI mean score differences noted, the initial 
CSAB and CBI mean scores showed no significant differences. 

The data analysis for the follow-up sample showed that the MSTOI 
mean scores were not significantly different for the initial and the 
follow-up samples. Therefore, (with the exception of Cecil County) 
comments concerning the follow-up sample can be generalized to the 
initial sample. 

The concurrent validity was determined for the MSTOI with the 
CSAB and the CBI for the combined Queen Anne's and Wicomico Counties 
sample. Both of the calculated values were significant at the .01 
level for the more than 600 score samples. The concurrent validity 
of the MSTOI was assessed further by determining the Pearson Product 
Moment Correlation Coefficients of the MSTOI with the CSAB and the CBI. 
The substantial correlations of .73 and .68 were interpreted as evidence 
that the MSTOI was measuring many of the same characteristics of student 
behavior as the other standardized measures. 

The predictive validity of the MSTOI with the PRS as a criterion 
measure was assessed with the computation was significant at the 
.01 level. When the predictive val idity of the MSTOI was assessed by 
the Pearson Product Moment Correlation Coefficient r value of .52 was 
obtained. There was no significant difference between the MSTOI 
correlation coefficient and the correlation coefficients for the CSAB 
and the CBI. 

A significant relationship was shown between the MSTOI scores with 
student grade placement. This significant relationship was additional 
evidence of the predictive validity of the MSTOI. ^ 
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For the combined sample, the item analysis data indicated that all 
of the Individual items had a significant relationship with the MSTOI 
total scores and with the PRS criterion measure. All subscale scores 
also were significantly related to the MSTOI and the PRS total score. 

SUMMARY 

This longitudinal study was undertaken to assess the predictive 
validity of the MSTOI and other initial assessment instruments. The 
initial assessments were administered to the children during their 
kindergarten year. Criterion measures were administered to a sample 
of these children when they normally would be completing the second 
grade. In the interim period, no special remediation was done based 
on the initial test scores. Predictive validity of the MSTOI was 
determined by examining the relationship between student scores on 
the initial assessment instruments and scores on the criterion 
measures . 

One of the outcomes of the study was the observation that the 
MSTOI has predictive validity for identifying students who have 
potential reading/learning problems. The predictive validity of 
the MSTOI compared favorably with the other initial assessment 
instruments. Predictive validity was determined by calculating 
contingency coefficients as well as by calculating Pearson Product 
Moment Correlation Coefficients. The contingency tables and the 
resulting indicate a significant - lationship between the kinder- 
garten MSTOI rating and the rating several years later. A note of 
caution: the magnitude of the value is related to the cutoff 
scores employed to dichotomize the score distributions. Cutoff scoves, 
however, are related to school practices, are flexible and are easily 
changed to identify a particular proportion of the sample. The Pearson 
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Product Moment Correlation technique was employed to calculate the 
degree of relationship between variable scores representing an 
underlying continuum. All Pearson Product Moment Correlation 
Coefficients indicating the relationship between the MSTOI and the 
other assessment measures were significant at the .01 level. 

In this study, a larger number of MSTOI scores were analyzed 
than were used in the initial norming study. Data analyzed here 
gives insight into the percentage of students being detected as 
needing further testing. 

Another outcome of the study was the development of predictive 
validity data regarding the MSTOI and its relationship with school 
progress and retention. It was observed that students scoring below 
the MSTOI cutoff were more likely to be retained in grade than 
students scoring above the MSTOI cutoff score. 

Further, the interrelationship of the MSTOI subscale scores with 
the MSTOI total score and criterion measures was calculated. MSTOI 
subscale scores were highly correlated with each other and with the 

MSTOI total score. 

The item analysis of the MSTOI using a comparatively large 
sample size showed a significant correlation of all items with the 
total MSTOI score. Thirty-three of the thirty-six items had a 
correlation of .5 or better. 

In addition to the predictive validity, the concurrent validity 
of the MSTOI was established by examining its relationship to the 
other initial assessment instruments. values and the Pearson 
Product Moment Correlation Coefficients were determined and all 
calculations were significant at the .01 level. 

The favorable showing of the MSiOI in the study affirmed that 
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teachers rating students with a rating scale can predict student 
progress as well as some individually administered instruments. 

Recommendation for Further Study 
The results from this investigation of the MSTOI answer some 
questions and suggests others. The following suggestions and 
recoimiendations are provided with the hope that there will be optimal 
utilization of the MSTOI in the EIIP. 

Although there was little evidence of teacher misscoring, 
considerations should be given to simplifying the MSTOI test form. 
The advantages and disadvantages associated with grouping items of 
the same subscale together should be investigated. The interpretation 
sheet could be simplified by eliminating the distinction between 
student scores of two and three during the observation. Since both 
the two and three are treated identically for indicating areas of 
further screening, the current system is unnecessarily complex. 

In future studies, it would be advisable to involve the 
evaluation consultants in the design of the study. Involving the 
evaluation consultants would simplify the transfer of data to 
computer format. Machine scoring of MSTOI evaluations should be 
considered. 

Future studies might focus on identifying and evaluating the 
progress of those students that score above the MSTOI cutoff score, 
but have a weakness in one or more of the areas of development. 
Intensive intervention with such youngsters may be the most effective 
use of educational resources. 

This study investigated the predictive validity of the MSTOI 
over a two year time span with subjects who were not exposed to 
intervention procedures. It might be quite instructive to determine 



73 



the validity of the MSTOI with students when intervention procedures 
were available. It would be instructive to determine the reliability 
of the MSTOI over time utilizing students in school systems that 
provide intervention procedures. 

There is a need to develop information concerning how the MSTOI 
scores are used by individual teachers for diagnostic purposes, 
individualizing instruction and as a basis for further testing. 
Although this study provided information concerning the validity 
of the MSTOI, it did not attempt to investigate the manner in which 
MSTOI scores are used or abused. 

From a pragmatic point of view, a survey of the successful 
Intervention methods used by teachers in the classrooms would be 
a small cost/good return study. It would be a source of information 
that could be used by teachers across the state. 

A longitudinal study of the relationship between MSTOI score 
and classroom grades might provide some information not now available 
when standardized tests are used as the criterion measure. 

A descriptive study documenting the proportion of students 
scoring above the MSTOI cutoff score, broken down by school system and 
year, could provide some useful information for those planning 
educational programs. 
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TABLE A 



Means And Standard Deviations Of MSTOI Scores For 
The Queen Anne's County Initial Sample {N=305) 







Standard 


Item No. 


Mean 


Deviation 


1 


3.24 


.80 
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3.44 


.99 
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4.72 


.47 


15 
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Sensory/ 
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Perception 
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Cognition 


37.72 


6.94 


Affect/ 






Motivation 


44.21 


7.89 


MSTOI Total 


145.13 


20.27 
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TABLE B 

Means And Standard Deviations Of CSAB Scores For 
The Queen Anne's County Initial Sample (N=305) 



Competency 
Areas 


Mean 


TTZ 1 J 

Standard 
Deviation 


Orientation 


37.52 


5.62 


Coordination 


y . /u 


1 Q7 


Discrimination 


26.71 


4.60 


Memory 


31.05 


6.88 


ComDrehension/ 
Concepts 


36.95 


7.59 


CSAB Total 


108.40 


17.04 




TABLE C 




Means And Standard Deviations Of CBI Sccres For 
The Queen Anne's County Initial Sample (N=308) 


Subscale 


Mean 


Standard 
Deviation 


Task Orientation 


14.02 


4.93 


Extraversion 


15.08 


4.49 


Prone to 
Frustration 


4.23 


3.80 


Total Score 


29.94* 


7.54 


*Prone to Frustration not included 


in total . 
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TABLE D 



Means And Standard Deviations Of PIAT Scores For 
The Queen Anne's County Follow-up Sample (N=80) 



Subtest 


Mean 


Standard 
Deviation 


Matnenatics 


on 1 0 


O . 00 


Reading 
Recognition 


32. 56 


1 AC 

7 .45 


Reading 
Comprehension 


31 .44 


8.02 


Spelling 


32.95 


8.17 


Gene^-al 
Information 


23.54 


9.77 


Total Test 


150.79 


33.24 




TABLE E 




Means And Standard Deviations Of The PRG For The 
Queen Anne's County Follow-up Sample (N=23C) 


Subscale 


Mean 


Standard 
Deviation 


Auditory 
Comprehension 


11.78 


3.12 


Spoken Language 


14.63 


3.63 


Orientation 


12.01 


2.72 


Motor Coordination 


9.10 


1.57 


Personal /Social 
Behavior 


24.77 


5.57 


Verbal Score 


26.00a 


6.83 


Nonverbal Score 


48.16b 


9.72 


Total Score 


74.16 


15.99 


^Verbal Score includes auditory comprehension and 



spoken language subscale scores. 
"^Nonverbal Score includes orientation, motor 
coordination and personal/social behavior subscale 
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TABLE F 

Means And Standard Deviations Of MSTOI 
Follow-up Sample for Queen Anne's County (N=230) 



1 tein no • 


Mean 


Standard 
Ppvi ation 
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3. 


25 


.82 
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3. 


46 


1 .00 
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12 
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19 
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3.41 


Language 


35 
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6.67 
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6.01 


Perception 


30 


.11 


Cognition 
Affect/ 


38 


.16 


6.48 


Motivation 


44 


.71 


7.30 


Total Score 


145 


.80 


20.60 
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TABLE G 



Means And Standard Deviations Of CSAB Competency Areas 
For The Queen Anne's County Follow-up Sample {N=229) 



Competency 
Areas 


Mean 


Standard 
Deviation 


Orientation 


37.90 


5.41 


Coordination 


9.83 


1 .93 


Discrimination 


26.93 


4.44 


Memory 


31 .45 


6.88 


Comprehension/ 
Concept 


37.40 


7.60 


CSAB Total 


109.64 


16.70 



TABLE H 

Means And Standard Deviations For CBI For Queen 
Anne's County Follow-up Study (N=230) 



Subscores 


Mean 


Standard 
Deviation 


Task Orientation 


14.45 


4.80 


Extraversion 


15.74 


4.46 


Prone to 






Frustration 


3.92 


3.57 


Total 


30.28* 


7.61 


*Prone to Frustration 


not included 


in total . 
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TABLE I 

Means And Standard Deviations Of MSTOI Scores 
For The Wicomico County Initial Sample (N=546) 



Item No. 


Mean 


Standard 
Deviation 


1 


3,46 




2 


3,59 




3 


4,04 
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4,60 


• / 1 
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4,04 


"7 71 

.74 
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3,70 


.95 
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4,34 


.81 
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4,68 


.73 
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3,99 


.87 


in 


4,01 


.93 


n 


3,86 


,84 


12 


A n 

4.57 


,bb 


13 


4.50 




14 


A ~I A 

4,74 




15 


4,20 


.y/ 


16 


A OO 

4,22 


.y 1 


17 


4,63 


71 
./I 


18 


4,16 


.y 1 


19 


A ^ r\ 

4,69 


, /b 


t.\i 


3,82 


,90 


21 


3,64 


,97 


22 


4,47 


,75 


23 


4,42 


,74 


24 


3,71 


,90 


25 


4,43 


,79 


26 


4,48 


,67 


27 


4,05 


,81 


28 


3,78 


.97 


29 


4,12 


.82 


30 


4,18 


,97 




H , H-y 




11 


4 37 


.82 


J J 


4 41 


.74 


34 


3,72 


1.13 


35 


3,48 


1 .11 


36 


3,52 


.97 


Psychomotor 


20,51 


3.45 


Language 


36,00 


6.62 


Sensory/ 




6.09 


Perception 


31 ,5] 


Cognition 


39,27 


6.34 


Affect/ 






Motivation 


44,93 


7.49 


Total Score 


149,10 


19.95 
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TABLE J 



Means And Standard Deviations Of CSAB Scores 
For Wicomico County Initial Sample (N=546) 



Competency 
Areas 


Mean 


Standard 
Deviation 


Orientation 


38.15 


5.59 


Coordination 


9.57 


2.17 


Discrimination 


27.00 


4.57 


Memory 


30.67 


6.46 


Comprehension/ 
Concept 


36.97 


7.89 


CSAB Total 


108.94 


17.55 



Note: Subscale total will not equal total since 

some items appear in more than one subscale. 



TABLE K 

Means And Standard Deviations Of CBI Scores 
For Wic'.Tiico County Initial Sample (N=546) 



Standard 

Subscal e Mean Deviation 



Task Orientation 


13, 


.92 


5, 


.58 


Extraversion 


r, 


.37 


4, 


.61 


Prone to 
Frustration 


6. 


.12* 


4, 


.98 


Total CBI 


28, 


.89 


8, 


.64 



*Not Included in computing total CBI. 



so 



TABLE L 

Means And Standard Deviations Of PRS Scores 
For Wicomico County Follow-up Sample (N=387) 







Standard 


Subscale 


Mean 


Deviation 


Verbal 


28.07 


7.27 


Nonverbal 


47.81 


10.53 


PRS Total 


75.88 


15.98 
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TABLE M 



Means And Standard Deviations For MSTOI Scores 
For The Wicomico County Follow-up Sample (N=386) 



Item No. 


Mean 


Standard 
Deviation 


1 


3.50 


.94 


2 


3.65 


.96 


3 


4.10 


.80 


4 


4.69 


.59 


5 


4.09 


.71 


6 


3.77 


.91 


7 


4.38 


.83 


8 


4.74 


.59 


9 


4.06 


.84 


10 


4.07 


.93 


n 


3.90 


.82 


12 


4.65 


.59 


13 


4.56 


.64 


14 


4.78 


.46 


15 


4.23 


.95 


16 


4.29 


.86 


17 


4.67 


.66 


18 


4.25 


.84 


19 


4.76 


.64 


20 


3.90 


.88 


21 


3.67 


.93 


22 


4.52 


.71 


23 


4.48 


.66 


24 


3.78 


.88 


25 


4.47 


.75 


26 


4.55 


.61 


27 


4.08 


.80 


28 


3.82 


.97 


29 


4.19 


.97 


30 


4.24 


.89 


31 




. OU 


32 


A A A 

4.44 


.76 


33 


A A ^ 

4.46 


. /O 


34 


O "70 




35 


3.51 


1.11 


36 


3.56 


.96 


Psychomotor 


20.98 


2.70 


Language 


36.75 


5.53 


Sensory/ 




5.03 


Perception 


32.19 


Cognition 


40.07 


4.86 


Affect/ 




6.17 


Motivation 


45.86 


Total 


151 .06 


18.3:< 
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TABLE N 



Means And Standard Deviations Of CSAB Scores 
For Wicomico County Follow-up Sample (N=385) 



Arp^s 



Competency Standard 



Mean Deviation 



Orientation 38.64 5.16 

Coordination 9.77 1-88 

Discrimination 27.45 4.21 

Memory 31.28 5.80 
Comprehension/ 

Concept 37.50 7.12 

CSAB Total 110.65 15.44 

N^te^ Subscale total will not equal total since 

some items appear in more than one subscale. 



TABLE 0 

Means And Standard Deviations Of CBI Scores 
For Wicomico County Follow-up Sample (N=386) 







Standard 


Subscale 


Mean 


Deviation 


Task Orientation 


14.26 


5.31 


Extraversion 


15.47 


4.62 


Prone to 
Frustration 


6.01* 


4.86 


Total CBI 


29.44 


8.28 



*Not included in computing total CBI. 
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TABLE P 



Means And Standard Deviations Of MSTOI Scores 
For The Cecil County Population (N=435) 



Standard 

MSTOI Item Mean Deviation 



1 


3.27 


.93 


2 


3.50 


.97 


3 


3.79 


.93 


4 


4.75 


.55 


5 


4.04 


.71 


6 


3.48 


.95 


7 


4.15 


.84 


Q 
O 


t . 04 


77 
til 


9 


3.84 


.90 


10 


3.93 


.94 


n 


3.69 


.91 


12 


4.21 


.91 


13 


4.25 


.83 


14 


4.58 


.67 


15 


4.19 


.94 


16 


3.98 


.90 


17 


4.59 


.83 


18 


4.10 


.89 


1 Q 


t . o . 


84 


on 


? 7d 


84 


21 


3.51 


.97 


22 


4.50 


.76 


23 


4.29 


,89 


24 


3.51 


.91 


25 


4.58 


.69 


26 


4.23 


.80 


27 


4.00 


.77 


28 


3.80 


1 .01 


29 


4.11 


.79 


30 


4.12 


.93 


31 


4.45 


.97 


32 


3.98 


.95 


33 


4.19 


.91 


3^ 


3.63 


1 .11 


35 


3.67 


1 .00 


36 


3.50 


1 .10 


Psychomotor 


20.14 


3.18 


Language 


34.81 


7.33 


Sensory/ 




6,00 


Perception 


31 .01 


Cognition 


38.38 


6.31 


Affect/ 




6-85 


Motivation 


44.29 


MSTOI Total 


145.00 


23.15 



TABLE Q 

Means And Standard Daviations Of MSSST Scores 
For Cecil County Initial Sample (N=88) 







Standard 


Subscale 




Deviation 


Motor Patterning 


17.18 


7.75 


Visual/Perceptual 
Motor 


16.22 


6.33 


Language 


15.23 


7.60 


MSSST Total 


48.43 


20.68 



TABLE R 

Means And Standard Deviations Of PIAT Test Scores 
For C6cil County Follow-up Sample (N=79) 



Item 


Mean 


Standard 
Deviation 


Mathematics 


32.29 


11 .85 


Reading 
Recognition 


34.27 


13.02 


Reading 
Comprehension 


32.78 


12.33 


Spel ling 


33.39 


12.51 


General 
Information 


28.17 


12.52 


Total 


160.84 


58.45 
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TABLE S 



Means And Standard Deviations Of PRS ?cor&?; 
For Cecil County Follow-up Sample (N.=79) 



Item 


Mean 


Standard 
Deviation 


Auditory 
Comprehension 


12.87 


4.49 


Spoken Language 


15.80 


5.47 


Orientation 


13.59 


3 


Motor 

Coordination 


10.39 


2.79 


Personal /Social 
Behavior 


26.85 


7.31 


Verbal Subtotal 


28.67 


9.79 


Nonverbal 
Subtotal 


50.71 


13.40 


Total 


79.38 


2a. 40 



TABLE T 



Means And Standard Deviations Of MSTOI Scores 
For The Cecil County Follow-up Sample {N=79) 



Standard 



Item No. 


Mean 


uc V 1 a L \\jm 


1 


3.11 


1.27 


2 


3.23 


1 .45 


3 


3.58 


1.31 


4 


4.54 


.84 


5 


3.92 


.84 


6 


3.27 


1 .30 


7 


3.92 


1 .09 


8 


4.14 


1 .13 


9 


3.56 


1 .21 


10 


6 . 


1 29 


n 


3.48 


1.27 


12 


4.03 


1 .25 


13 


4.09 


1 .04 


14 


4.44 


.90 


15 


4.20 


.94 


16 " 


3.67 


1.26 


17 


4.30 


1 .16 


18 


3.89 


1 .18 


19 


4.11 


1.34 


20 


3.49 


1.15 


C. 1 


3 33 


1 .3' 




4 43 


.84 


9'K 


3 99 


1 .21 


9L 


3 24 


1 .23 


9^ 


4 44 


.87 




4.09 


1 .09 


27 


3.92 


.84 


28 


3.49 


1 .41 


9Q 


4.01 


.99 


30 


3.97 


1 .17 


31 


3.97 


1 .48 


32 


3.72 


1 .39 


33 


3.82 


1 .29 


34 


3.37 


1 .46 


35 


3.39 


1 .41 


36 


3.20 


1 .49 


Psychomotor 


18.81 


4.73 


Language 


32.48 


10.75 


Sensory/ 




8.67 


Perception 


22.07 


Cognition 


35.83 


9.42 


Affect/ 




9.12 


Motivation 


42.42 


MSTOI Total 


136.01 ' 


36. CI 



tAble u 

Means And Standard Deviations Of The MSSST 
Scores For Cecil County Follow-up Sample (N*79) 



Subscale 


Mean 


Standard 
Deviation 


Motor Patterning 


14.47 


9.36 


Visual/Perceptual 
Motor 


13.58 


7.97 


Language 


12.73 


8.76 


MSSST Total 


40.77 


25.13 



TABLE V- 



Means And Standard -Deviations Of MSTOI Scores 
For The Combined County. Initial Sample {N=128 



Item NO. 


• ricai 1 


Standard 
Deviation 


1 


.. 3.34 


.93 


2 


3.52 


QQ 

.yy 


3 


4.93 


0"7 

.8/ 


4 


4.61 


CO 

.bo 


5 


4.06 


70 


6 


3.63 


n o 

.9J 


7 


4.27 


.81 


8 


4.57 


O 0 


9 


3.38 


.90 


10 


4.00 


.91 


n 


3.76 


.87 


12 


4.39 


7Q 

. /y 


13 


A AC 

4.4b 


7^^ 


1 A 

14 


A CO 

4. bo 


. DO 


15 




. y D 


16 


A no 

.uy 


. 


17 




77 

mil 


18 


A in 
4.19 


.0/ 


19 


/I o 


.OU 


20 




.OD 


21 


3.57 


.95 


22 




7fi 


23 




.0 \ 


24 


O CO 

J. bo 


. ^ J 


25 


/I /1 7 


• / 0 


26 


yi on 
4. J9 


• 


27 


4 .Ub 


• OU 


28 


J .o 1 


Qn 


29 


4.17 


.77 


30 


4.14 


.95 


31 


4.44 


.90 


32 


4.19 


.86 


33 


4.36 


.79 


34 


3.55 


1 .14 


35 


3.53 


1 .07 


36 


3.47 


1 .04 


Total 


146.77 


21 .25 
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Description Of Assessment Instruments 
The five initial assessments used were the Maryland Systematic 
Teacher Observation Instrument (MSTOI), the Classroom Behavior 
Inventory (CBI), the Self-Control Behavior Inventory (SCBI), the 
Cognitive Skills Assessment Battery (CSAB), and the Meeting Street 
School Screening Test (MSSST) . The first three are teacher rating 
scales requiring teacher observation and no student responses. The 
last two require student responses. A brief description of each 
initial assessment follows. 

Maryland Systematic Teacher Observation Instrument . Each child is 
assessed on the 36 items of the observation instrument. Each behavior 
is rated on a scale frequently, often, sometime, seldom, or never. 
Numerical scores range from one to five per item with a possible 
maximum total score of 180. Individual items are grouped under 
five subscales: psychomotor (3 items), sensory perception (8 items), 
language (9 items), cognition (9 items), and affect motivation (14 
items). Several items are included under more than one subscale. 
Students with a total score of less than 139 were designated as 
being high risk students having potential reading/learning problems. 
This instrument was developed by the Maryland State Department 
of Education in cooperation with local learning agencies. 
Classroom Behavior Inventory . The child is assessed on the 15 items 
that constitute this inventory. Each item is evaluated as belonging 
to one of seven categories: never, almost never, occasionally, half 
the time, frequently, almost always, or always. Numerical values of 
0, 1, 2, 3, or 4 are assigned to these categories by equating the 
zero value to never and almost never while the two highest categories 
almost always and always are given in numerical value of four. Maximum 
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score is 60. Individual items are grouped into three subscales: task 
orientation, extraversion, and prone to frustration. Only the first 
two categories are included in computing the total score. For this 
study the cutoff score of 32 was established. Any student scoring 
less than 32 was identified as having potential learning/reading 
problems. This instrument was developed by Schaefer, Aaronson, and 
Small of "the Montgomery County Public School System, Rockville, 
Maryland. 

Self-Control Behavior Inventory . Each child is assessed with the 
eight items on this inventory. Each item is scored as rarely does, 
sometimes does, usually does, and almost always does. Numerical 
scoring of the items ranges from zero to three.. Maximum test score 
is 24. For this study, a cutoff score of 15 was utilized. Any 
student having a total score of less than 15 was identified as having 
potential Teaming/reading problems. The test is available from . 
Psychoeducational Resources, Inc. in Washington, D. C. 
Cognitive Skills Assessment Battery . An individually administered 
battery requiring 20-25 minutes to administer. Information concerning 
the child's- abilities in the five competency areas of orientation 
toward and familiarity with one's environment, coordination, discrim- 
ination, memory and comprehension and concept formation are determined. 
Items and competency areas wer'e developed following examination of 
curricula currently followed, teacher interviews, and research in 
areas of primary school skills. Although the test developers did 
not provide a mathematics total score, one was used in this study. 
A previously developed scoring system, established a maximum score 
of 150 for the battery. A cutoff score of 128 was established to 
identify high risk students that have potential learning/reading 
problem. ThP Cognitive Skills Assess t Battery was developed by 
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Ann E. Boehm and Barbara Slater. The battery is available from the 
Teachers College P-ess, Columbia University. 
Meeting Street S ch ool Screening Test . This is an individually 
administered scrt ing isst developed to idBniify children that may 
have learn-ing disAjTlir^es. The test requires aporcximateTiy "5-20 
minutes to admin^^iter. Three subtests compromise the test: rotor 
patterning, vist^l/ps-^^^stual /motor and language subtests. ; lidity 
data are includec in -n?. dnual where the relatimship of M S~, to 
Illinois Test of"i^^yc:rr" -^nyuistic Abilities and xhe Frosti l)e»fel- 
opmental Tests r Visuaj. PerceEiion is presented. The test w«s 
normed in 220 kTr.-H::is~ten and 274 first gr^de wungsxers ^"^re 
test is publisriec r the ^teeti'^i Street Sc iOi;! , ^rovidenc Rtmde 
Island. 

The two vrit-fy assure- isd in the study were xhe --.eafaody 
Individual Ac e> ^t ~5sx (PlAT and the i^ipil RafcirTm i;;^ag (PRS). 
The former is VMually ac: "■?ribterr-_ -^st, ^.nd tt»e 1at~3r is a 

5rou3 test. 

? ^dy Individu gr' -ev 'ei.iert lezz . Tht . - a wiee-ni- '^^ -.cr-sning 

^._ure of ach"* 'e^ -in the areas of - lattenatic-s, read-ng, spelling 
wDEt general intt»nwcr.t,v . This iraividusVi administered unti«d test 
jsually require- ''^-- '-^ -linutes tc admiinister and to score tne battery. 
The subtests are d»s3ig"^d so that no academic skills are reauired 
other than those specifically being meesur^. No writing is required 
of the subject, "ssn- ,;oring is comple teifu/ objective. The PIAT, 
copyrighted in 19 ":;i3 available f ram t:1« American Guidance Services, 
Inc. 



PuDiil Rating Scale . This rating scaTe developed on ^the hypothes" 
thc if areas of deficit are '^arirfu] o 'iefinec and a-elineat3d, 
the? can be observed and rate^*: by regula:r classrsDovi teachers who 
are it\ close contact with chi ^ldre i. The classnjcir, teachers 
observe and rats c' ildren in ttie ^five behavior arss: audrd:ory 
comErrehension, spKKfen languag^e, orfientfftnon, roratn- coordination, 
ami personal /socia*-' -^ehavior-. The twenty-four :tian rating scale 
is cCTistructed ir ^ nanner such tlret each behavitsr is rankei on a one 
tc fi-*^e scale. T -ta 1 end suibscale scor^ are calralated. he PRS 
wais copyrighted ri 197^ and is aviilaMe from Gruee and Str-tton. Inc. 
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Technical Notes 

When confronted with a data analysis problem, the analyst must 
decide which statistical procedures should be utilized. The decision 
is based on the quality and quantity of the data. For this study, the 
x2. Phi coefficient and the Pearson Product Moment Correlation Coef- 
ficient were the principal statistical techniques employed. The data 
developed enabled the evaluators to make inferences about the probability 
that observed differences or relationships were due to factors other 
than chance. 

The X2 technique is most commonly usee as either a test of relErion- 
ship or as a goodness of fit test. The gooriness of fit approach assir^es 
some theoretical score distribution such as a normal curve. A signt-:cani 
x2 value indicates that the observed distribution does not fit the 
theoretical distribution. The used as a test of relationship t£Sts 
the null hypothesis of no relationship between two variables. A 
significant X^ value allows one to infer that there is a significant 
relationship between the variables under study. The latte- approach 
appeared to be better suited to the needs of this study. When is 
used as a test of statistical significance it tests for the Existence 
of a systematic relationship between two variables. 

The X^ technique most frequently used in this study was the X- 
corrected for continuity. This correction is especially appropriate 
when analyzing a 2 x 2 contingency table with N greater than 40. 
It also has the "advantage of incorporating a correction for 
continuity which markedly improves the approximation of the distribution 
of the computed X^ to the chi-square distribution." (Siegel) When 
the sample is very large, even miniscule deviations will generate a 
statistically significant chi-square. This is because larger samples 
are much more likely to approximate the true relationship in the 
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universe. In the oth ' ^and, a small sample is much more likely to 
contain a- sproporttffirat, .. number of atypical cases. 

If rn relations^- -rrsts between two variables in the sample 
under stcai: , then an; -stions from the expected values which occur 
ir a tab'« iased m - rt'^xiy selected samole data are due to chance. 
Wrrrle some ami 1 dev^^on reasonably car be expected due ro chance, 
U'.-ne deviariTjns-- large values of cn1-square, are un' ikely. 

S-nnra tee Ertua' r= tianships in the universe ar» unk<now!T. small 
va ..^s jf X- arfr i n^.^nreted to indicate tsie absence crF s relation- 
sh~, often '■^= ^r ts as statistical intfe^endence. lonversei, , - 
Is— chi-sou-f^'^e -rr that a systematic relationssiip of -t 

SO" -_rX i s ts b ^ twe»n " tte war i a b 1 e s . 

ly itseK, : ■^^-'Hnaffire value helps in making the decis^ox 
whetfier th: vdri^D s are independent or related. It doss ncn- 
indT- :te tms tr=»gtr of the relationship. The magnitude ot^ the 
X2 13 influieneed a' 'irie factors sample size and contingertcr table 
con=guratMOT- h -trtistical technique which adjusts for these 
fattnrs is pn* . 

For a I X Z to les, the phi stat->,tic is a suitable measure of 
associatiTjriK i.e. ^- neasure of strength of relationship- Phi makes 
a co^recti'"' for f^e fact that the value of chi-square ^s directly 
:prr :rtionu»l to the number of cases N by adjusting the X^. Phi 
.sa -'^on tjre value of 0 when no relationship exists, san: the value 
0 • -/hen the variables are perfectly related, i.e. al cases fall 
jm.t jn tae main or the minor diagonal. 

-he Pearsor Product Moment Correlation provides a nngle number 
which summarizes the relationship between two variables.. These 
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correlation coefftcien-s indicate the degree to wicr variation (or 
change) in one variable is related to variation (-nange) in another. 
A correlation coefficient not onl- summarizes the strength of 
association between a oair of variables, but also pr: -ides an easy 
means for comparing the strength of relationship betsen one pair 
of variables and a different pai • Of course, this technique is 
limited to interval and ratio lev^T variables. 
Statistical Procedu- ^ rs and CompjitBr Programs 

The evaluation consultants ^oyed the "Statistical Package for 
the Social S-ence? (SPSS) as rne scurce of the ccsnputer programs 
needed to atn^ vz.? data. SPSI is r integrated system of computer 
programs des:^^t fcrr the analy?" c- ^ta in the social sciences. 
It provides roE number of statistical routines and a good system 
for data trr- srt Tiation and file me ipulations. The SPSS package 
provides ir s" le and convenient inffnner a comprehensive package 
that allow the ^searcher to util^ many different types of data 
analysis ^ Tcludir i chi-square ana" s of variance, correlations 
and t-tes- Th package is one ^ the most widely used packages 
of statis al i-icedures used ir c country. 
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